OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Programming => Topic started by: Martin Vieregg on December 30, 2019, 01:02:08 am

Title: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on December 30, 2019, 01:02:08 am
One year ago, I wrote about my desire to get a substitute for the limited VIO window when using cmd.exe or 4os2.exe without 8000 character limit and with all editor functionality.
https://www.os2world.com/forum/index.php/topic,1859.msg18588.html#msg18588 (https://www.os2world.com/forum/index.php/topic,1859.msg18588.html#msg18588)

The last year, I was very busy with other things, but meanwhile I began to program. I am the author of ME Editor (http://www.hypermake.com/english/betatest.html#hd117) and other editors. The editor is an OOP (wdsibyl) project where I can easy manage modifications.

I managed now to excute the original cmd.exe with pipes for stdout, stdin and stderr. The basic input/output functionality works already. But  I need your help with some sophisticated problems.

The editor gets first executed, starts the pipes and executes cmd.exe in the background via
cmd.exe /K >outpipe 2>errorpipe <inpipe

For specific commands like mode, cls the commands will be interpretated by the editor and not cmd.exe, or gets interpretated by both. The current directory will be hold both in the editor and in cmd.exe. I will manage to turn the 4os2 escape codes into colors. All these things seem to make no problems to program (all not programmed yet).

But for three problems I have no solution yet:

(1) While cmd.exe is busy, the editor has to turn into read-only mode. The time period begins with an <ENTER> after typing the command, so the editor knows this moment. But when is the job finished and cmd.exe turns to idle state, the editor should turn to read/write and should accept the next command. I have no idea how the editor can get the information that cmd.exe is in idle state again.

A dirty solution would be to wait for [C:\] or a simular string. But this is not possible. And this is the next problem.

(CHANGED: Problem (2) is non-existant. It was a bug in my editor code.)
(2) A stdout output of cmd.exe waits until the line is finished with an carriagereturn. This has got a strange effect to my program: When starting the program, I do not see [C:\] at the moment. Then I type my command, e.g. "dir", press ENTER and the editor shows the "dir" output. But the last line with the command prompt [C:\] is not shown. It is only shown after entering the next command. Programs with an success indication with dots ...... are invisible while exeuting and waiting. So my question is: Is it possible to force stdout output in cmd.exe when the line is not terminated by a carriagereturn? Or do you have another idea how to sailing round the problem? The only idea for me at the moment is to simulate this output by adding a prompt and to swallow the first line output. Not a pretty solution. And the success dot problem is still not solved.

(3) Not all textmode programs do write into stdout, but there is text output shown nevertheless. This output gets shown in the hidden cmd.exe VIO window and not in my editor window. Can someone explain what this means and perhaps how to get this text in stdout nevertheless?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Pete on December 30, 2019, 01:43:49 am
Hi Martin

For problem 1 could you not start a thread to execute cmd.exe and monitor that thread? - DosCreateThread, DosWaitThread maybe...


Regards

Pete
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Rich Walsh on December 30, 2019, 08:38:53 am
(3) Not all textmode programs do write into stdout, but there is text output shown nevertheless. This output gets shown in the hidden cmd.exe VIO window and not in my editor window. Can someone explain what this means and perhaps how to get this text in stdout nevertheless?

A VIO window is not a virtual TTY device. It's an emulation of a full-screen text mode session. In these sessions, the screen is accessed via the Vio API functions which can write anywhere on the screen independent of stdout/stderr. When an application like cmd.exe wishes to write stdout/stderr to the screen, it probably uses an internal version of VioWrtTTY().

To capture Vio API output, you would need access to the actual screen buffer used by full-screen apps or the virtual screen buffer used by VIO windows. Unfortunately, these buffers belong to other processes, so your app has no way to get at them.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on December 30, 2019, 11:18:29 am
Meanwhile, I have changed the first post: Problem (2) was my failure. The editor has read the input data line by line. This is now fixed.

Problem (3): I remember that in former times, I have programmed text applications where I used a "goto" command. Of course this is not stdout. I fear that this is a general limitation of my program. Perhaps I can add a list of these programs and then a conventional VIO window gets started, or I define a command /VIO where the user can start a VIO window in a command.

Problem (1): Wenn running cmd.exe in a thread (I have got already an input thread in my editor because waiting for input blocks the process),  I still have no access to cmd.exe. But it would be possible to restart cmd.exe for every command the user types into the editor in a specific execution thread. While cmd.exe is running, the thread is frozen, and after finishing cmd.exe, the thread code continues. This could be a clean solution. The current directory is only hold in the editor and every time I execute cmd.exe, I have to hand over the current directory. But this should not be a problem.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Neil Waldhauer on December 30, 2019, 03:50:33 pm
I think Vio output is not defined for a larger screen window. So to really do this right, you would need to extend the Vio family. But if some programs write to the buffer themselves, you would need to monitor that, too.

Both of these things could be done by code inserted into the program's space using the registry entry User->SYS_DLLS->LoadPerProcess. You write your code into a DLL and gain control using the DLL initialization code.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Rich Walsh on December 30, 2019, 06:01:34 pm
User->SYS_DLLS->LoadPerProcess

This only applies to PM apps. None of the DLLs listed in SYS_DLLS are loaded into VIO apps.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Bogdan on December 30, 2019, 11:44:20 pm
Perhaps I can add a list of these programs and then a conventional VIO window gets started, or I define a command /VIO where the user can start a VIO window in a command.
This may only work for programs that don't use the following calls:
Most likely, session management calls are affected also.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Alex Taylor on January 01, 2020, 02:56:06 am
Maybe a dumb question, but have you taken a look at KShell?  If I understand what you're describing, KO seems to have done something similar there.
http://hobbes.nmsu.edu/h-viewer.php?dir=/pub/os2/util/shell&file=kshell081.zip
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 02, 2020, 01:27:24 pm
kshell has no editor functionality. The main interest is to have full editor functionality, to resize the window and having an endless buffer to restore old outputs.

Meanwhile, the basic functionality works. I want to implement now colors with ESC codes. Where is a documentation about VIO ESC codes?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 02, 2020, 04:37:25 pm
They're just standard ANSI colour codes and seem to work the same on most all platforms so should be lots of documentation around, just search for ANSI Escape Code.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 02, 2020, 04:46:05 pm
Have you looked at x86sup.sys ftp://ftp.netlabs.org/pub/xfree86/sources/xf86sup.zip (http://ftp://ftp.netlabs.org/pub/xfree86/sources/xf86sup.zip) ? It allowed creating PTTY's and TTY's and was used for the xterm in XFree/2. Unluckily it is a device driver, guess needed to be to access the VIO buffer.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 08, 2020, 10:12:46 pm
Meanwhile, I could program the basic functionality. It basically works fine already. I needed to program several threads: One for stdin, one for stderr, one for cmd.exe itself. stdout commands (the commands the user enters) are sent via main thread. It was only a few hundred lines of new code, but very tricky and nervy. It works now both with cmd.exe and 4os2.exe. Colors are already working. The problem with the unknown output end of a cmd.exe command is not a real problem. The user always can enter input while cmd.exe output is running but that doesn't matter because noone will do so. The problems above are solved. The next jobs is to implement a selection of editor functionality which I have already programmed for ME Editor which will be quite easy.

But I didn't manage to send a Ctrl-C character/message to cmd.exe to stop execution of a current command output. I tried ASCII #3 and ESC[Cm and ESC#3. Any idea how to send the break command to cmd.exe /4os2.exe? Has anyone studied the 4os2 source code?

Then I wonder if the fine working command to execute cmd.exe via my program
program: 'cmd.exe'
parameters: '/C cmd.exe /K <\pipe\mshelloutpipe >\pipe\mshellinpipe 2>\pipe\mshellerrpipe"

which starts cmd.exe twice is really necessary. Any ideas? Can I use pipenames in the OS/2 STARTDATA structure?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Andreas Schnellbacher on January 09, 2020, 12:54:10 am
But I didn't manage to send a Ctrl-C character/message to cmd.exe to stop execution of a current command output.
Not knowing any details, I can only say that there exists something. The EPM command line window uses SUE_break of ETKR603.DLL. Unfortunately it's closed source. Other functions are SUE_new, SUE_free, SUE_write, SUE_readln. I don't even have a clue what 'SUE' means.

After having found a function, you probably have to assign a key def for Ctrl+Break (and Ctrl+C) to that, either WM_Char or accelerator.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 09, 2020, 01:17:15 am
Wonder if sending a SIGINT would work? That's what CTRL-C sends in most operating systems. SIGBREAK for CTRL-Break
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 09, 2020, 01:27:18 am
The function to send the break signal to a process is DosSendSignalException, using the pid of the process and XCPT_SIGNAL_INTERRUPT  or XCPT_SIGNAL_BREAK as the second argument. The Ctrl-C character combination or scancode(3) won't work. Ctrl-C is intercepted by the OS and a signal is sent to the process that has focus. You'll need to set a signal handler for the signal in your main process and within that send the signal to the child process using DosSendSignalException.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Andi B. on January 09, 2020, 08:06:52 am
For the redirection I think you can do this in your program. DosCreatePipe and friends may help. I don't have exactly what you need but maybe this snippset gives you a starting point -

Code: [Select]
DosDupHandle( HF_STDOUT, &hfSave);        /* Saves standard output handle      */
DosCreatePipe(&hpR, &hpW, PIPESIZE);      /* Creates pipe                      */
DosDupHandle(hpW, &hfNew);                /* Duplicates standard output handle */
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 09, 2020, 11:07:11 pm
Quote
The function to send the break signal to a process is DosSendSignalException, using the pid of the process and XCPT_SIGNAL_INTERRUPT  or XCPT_SIGNAL_BREAK as the second argument

Thank you for this useful hint. But it does not work. I get the Session PID from DosStartSession (last parameter). DosSendSignalException is running from another thread (because DosExecPgm blocks the thread). DosSendSignalException returns 205  ERROR_NO_SIGNAL_SENT.

Any ideas?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 10, 2020, 12:01:37 am
The 205 error meant that there isn't a process with a signal handler in the process tree. You should set  up an exception handler in the thread that calls DosExecProcess using DosSetExceptionHandler prior to issuing the DosExecProgram
You can run the process asynchronously by passing EXEC_ASYNCRESULT or EXEC_ASYNC as the third parameter in the DosExecPgm call. The latter flag discards any result from the called process.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 10, 2020, 02:07:41 pm
It still doesn't work even I install an exeption handler for the process. I still get error 205. I run the program asnychrous. The thread rests at DosExecPgm while the program is running and continues when exiting the program.

Could it be possible that only the first cmd.exe which exeuctes the second is in reach for me? I execute cmd.exe first and this cmd.exe executes a second cmd.exe and sends the pipenames as parameters:

program: 'cmd.exe'
parameters: '/C cmd.exe /K <\pipe\mshelloutpipe >\pipe\mshellinpipe 2>\pipe\mshellerrpipe"

If this is a problem, I have to start cmd.exe directly. But how can I hand over the pipe names to the program/process? Via STARTDATA parameter? I haven't found a solution.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 10, 2020, 03:19:42 pm
Yes, you don't have the pid of the second cmd process and there's no simple way to get hold of it. .

You don't need to pass the pipe name to the child process. A child process inherits the open file handles of the parent. Create a pipe in the parent process, Save the parent processes stdin and stdout to temporary file handles and call DosDupHandle on stdin and stdout setting them to the write and read handles of the pipe. Then call DosExecProgram and following that restore stdin and stdout in the parent. At that point anything writtten to the pipe's write handle from the parent should go to stdin for the child and anything written to stdout in the child can be read from the pipe's write handle.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 10, 2020, 04:58:52 pm
>Create a pipe in the parent process

OK. That has been done from the beginning of my project.

>Save the parent processes stdin and stdout to temporary file handles

OK. I have them via DosCreateNPipe parameter.

>and call DosDupHandle on stdin and stdout setting them to the write and read handles of the pipe.

I assume that the Outpipe of my program is the stdin of the child program and reverse. So I have to reverse the assignments in DosDupHandle.

>Then call DosExecProgram and following that restore stdin and stdout in the parent.

I read the docu of DosExecPgm and indeed, I found:

Quote
The new process inherits all file handles and pipes of its parent, although not necessarily with the same access rights:

- Files are inherited except for those opened with no inheritance indicated.
- Pipes are inherited.

A child process inherits file handles obtained by its parent process with DosOpen calls that indicated inheritance. The child process also inherits handles to pipes created by the parent process with DosCreatePipe. This means that the parent process has control over the meanings of standard input, output, and error. For example, the parent could write a series of records to a file, open the file as standard input, open a listing file as standard output, and then execute a sort program that takes its input from standard input and writes to standard output.

Because a child process can inherit handles, and a parent process controls the meanings of handles for standard I/O, the parent can duplicate inherited handles as handles for standard I/O. This permits the parent process and the child process to coordinate I/O to a pipe or file. For example, a parent process can create two pipes with DosCreatePipe requests. It can issue DosDupHandle to redefine the read handle of one pipe as standard input (hex 0000), and the write handle of the other pipe as standard output (hex 0001). The child process uses the standard I/O handles, and the parent process uses the remaining read and write pipe handles. Thus, the child process reads what the parent process writes to one pipe, and the parent process reads what the child process writes to the other pipe.

>At that point anything writtten to the pipe's write handle from the parent should go to stdin for the child and anything written to stdout in the child can be read from the pipe's write handle.

I will try it and report here.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 10, 2020, 06:16:27 pm
>OK. I have them via DosCreateNPipe parameter.
No, you haven't. You need to save the current stdin and stdout to  temporary file handles, your main process is a PM application, stdin and stdout are not necessarily what you think they are in that context, they're certainly not connecte to a console and you'll need to restore them at the end of this process. If you use a named pipe you don't have a file handle you have a handle to a named pipe. They're both unsigned longs, but they point to completely different structures. I suspect that the outcome will be tears before bedtime. Use an unamed pipe, you get two file handles a read handle and a write handle on the return from that call.

>>and call DosDupHandle on stdin and stdout setting them to the write and read handles of the pipe.

>I assume that the Outpipe of my program is the stdin of the child program and reverse. So I have to reverse the assignments in DosDupHandle.

No, your child process inherits the parent processes stdin and stdout which you set to the read and write handles of the pipe using the DosDupHandle call. Don't reverse them, you dup the read file handle with your parent process stdout so that the parent processs standard output is now the write filehandle of the pipe and then dup the read handle with the read file handle. Following this the parent process's standard input and standard output are the read and write filehandles of the unnamed pipe respectively. When your child process inherits the parent't processes stin and stout they'll be correctly linked to the appropraite ends of the pipe.

Following the call to DosExecProgram restore the parent process stdin and stdout from where you saved them at the start of this proces.
Now, when the parent process writes to the write file handle of the pipe it will go to the stdin of the child process because the other end of the write filehandle in the parent is connected to the read filehandle in the child and vice versa.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 10, 2020, 06:48:32 pm
If I understand it right,

- I have to use DosCreatePipe (an unnamed pipe) three times for stdin, stdout and stderr (but at this moment, they are not assigned yet) and I will get real file handles, 3 are used, 3 are not used. Or do I only use one Pipe with read and write? (But where is stderr then?)

- I call DosDupHandle and assign them to stdin, stdout and stderr.

>you dup the read file handle to your parent process stdout
but that means reverse.

When terminating the program, what do I have to do ?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 10, 2020, 08:24:11 pm
No you need to create one pipe for stdin and stdout. You provide three parameters to the DosCreatePipe, Two HFILE pointers and the size of the pipe. Which one you designate the read handle and the write handle is arbitrary, what get's written to one end of a filehandle can be read at the other, so if you dup stdin with the first handle then your child process will inherit that connected to it's standard inout, so whatevr the parentr process writes to the first filehandle will be read by the standard input of the child, the process for the second file handle is similar, duping it with stdout so that it's inherited as the stdout of the child process. The parent can then read from that file handle anything the child writes to it's standout. Data can travel both ways through each filehandle, however coordinating reading and writing to a signel channel would be complex and error prone, hence the abstraction provides two channels and you use one for reading and one for writing.  If you want to connect stderr to a pipe you'll need to create a second pipe, dup stderr to one of the filehandles and use that filehandle tin  the parent to read the output of stderr in the child. If you're being tidy you call DosClose on the other filehandle.

So, each call to DosCreatPipe gives you two file handles.

In the parent
1. ***Save your stdin and stdout***
2  Dup one filehandle with stdin, one with stdout, The parent process's stdin is now the filehandle from the pipe. Ditto for stdout. The parents stdinput is connected to the filehandle in the pipe, the child process inherits that filehandle as it's standar output, so that's where it reads from.
3. Call DosExecProg, the child inherits the parents stdin and stdout which are the two filehandles returnd from the pipe call.
4. In the parent
***restore the saved stdin and stdout***
 So stdin in the parent is not stdin in the child. The parent writes to the filehandle that's connected to the child processes stdin. 

In the child
1 The filehandle that was returned in the DosCreatePipe call in the parent is inherited as stdin, the OS tells the child it's value is 0.
2. The other filehandle returned by the call to DosCreateFile in the parent is inherited by the child as it's standard output. The OS tells the child that it's value is 1.

The child reads from the filehandle with the value 0 and writes to the filehandle with the value 1. The parent reads and writes to the pipe, it's restored it's standard input and standard output to their original values after the call to DosExecProgram.

stdin and sdout are abstractions provided by the operating system. Every process reads from fd0 and writes to fd1, The OS  juggles what they actually point to behind the curtain. The parent process changes what it's stdin and stdout point to from fd0 and fd1 to whatever value is returned in the call to DosCreatePipe, the child inherits those values as it's stdin and stdout and the parent resores it's original stdin and stdout. So the parents stdout and stdin aren't coinnected ot the childs, the filehandles returnd from the call to DosCreatePipe are.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 12, 2020, 10:30:51 am
Thank you for the hints, Laurence. I finished now programming (the length of the code is less now), it seems to be quite simple. After a second try, it works yet.

The main progress I made by studying a sample code popen.c (https://fossies.org/dox/cvs-1.11.23/popen_8c_source.html)

A pipe has got two ends: one for reading and one for writing. (Laurence told us that the reverse direction is also possitble.) So what to do is

1. DosCreatePipe with two handles:  A "write-end" and a "read-end" for the pipe. So a pipe is always defined by a pair of file handles. The "write-end" is for the parent program and the "read-end" for the child program when sending data from parent to child (the user enters command input in my parent editor) or reverse for sending cmd.exe output into the editor. I need three pipes for handling stdin, stdout and stderr.

2. To tell the child the correct filehandles for the child-end of the pipe, I need to use DosDupHandle. The child-end file handles are duped to the filehandle values 0, 1 2 for stdin, stdout and stderr. When using DosExecPgm to start the child program, these handles get available for the child.

Restoring and Saving the file handles with additional DosDupHandle calls are not necessary in a PM app, but can be done.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 13, 2020, 11:48:54 pm
After I managed the pipes, the Ctrl-C break is still not working. At the moment, I generate a thread where cmd.exe gets executed synchronous. While running cmd.exe, the thread "hangs" in the DosExecPgm function call. The necessary DosSendSignalException call when a user in the parent program editor enters Ctrl-C needs a process identification (PID). But I have only a thread identification (TID) from my thread. How do I get the PID ? Do I have to run cmd.exe asynchronous?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 14, 2020, 02:43:58 pm
According to the toolkit documentation for DosExecPgm
Quote
If asynchronous execution is indicated, DosExecPgm places the process ID of the started child process into the first doubleword of the pRes structure. If EXEC_ASYNCRESULT is specified for execFlag, the parent process can issue DosWaitChild (after DosExecPgm) to examine the result code returned when the child process ends. If the value of execFlag, is EXEC_ASYNC, the result code of the asynchronous child process is not returned to the parent process.
If the process is run synchronously the codeTerminate field of the RESULTCODES structure contains the result code returned on completion of the process. When the process is run asynchronously the  codeTerminate field contsins the pid of the process. So to get the pid you need to run the process ansynchronously, and, if you want both the pid and the resultcode you have to run it asynchronously and call DosWaitChild.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 14, 2020, 05:08:50 pm
OK, now the child (cmd.exe) runs asynchrously and I exeute DosWaitChild after starting cmd.exe by using DosExecPgm. All functions return no error. The returned PID is zero (is that normal?) The program still runs fine with the modifications.

Because my parent program is an editor which already has a function for pressing Ctrl-C, I don't think I need to install an exception handler in the main process. If the exception handler is necessary nevertheless, what do I write into the @function which is related to the exception?

Although I have now the PID, DosSendSignalException (PID, XCPT_SIGNAL_INTR) has no effect. The function is definitely executed in the main thread when pressing Ctrl-C in the parent editor. It still returns the value 205 (no process or no process with exception handler). Any ideas?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 14, 2020, 08:02:03 pm
A returned pid of zero isn't normal. You can check  what pid you should be getting by running top and looking for the cmd process you launched. If you're using  zero as the pid in your call to DosSendSignalException then the Exception will not be sent to your process. There isn't a process with a pid of zero, SYSINIT is the base process and its pid is 1. So that's why you're getting a 205 error, there's no process with that pid.

The PID of the process will, if the EXEC_ASYNC flag is used, be in the RESULTCODES->codeTerminate field when the call to DosExecPgm returns. Are you saving the value before you call DosWaitChild? DosWaitChild will set that value to zero for a normal exit.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 14, 2020, 10:07:04 pm
Oops, that was my failure. The return value has been set to another value later. Now the PID which I save corresponds to the TOP value. But DosSendSignalException still returns 205. My program log sends:

2020.1.14 21:40:55.699: exec DosExecPgm
2020.1.14 21:40:55.699: cmd.exe has been started. child_PID 213
...
2020.1.14 21:40:59.900: ScanEvent Ctrl-C, PID 213
2020.1.14 21:40:59.900: DosSendSignalException returns 205
2020.1.14 21:41:0.369: ScanEvent Ctrl-C, PID 213
2020.1.14 21:41:0.369: DosSendSignalException returns 205
2020.1.14 21:41:1.900: ExternProgThread still running
(User ends program)
2020.1.14 21:41:1.900: Bytes read 0
2020.1.14 21:41:1.900: not finished
2020.1.14 21:41:1.900: End cmd.exe child_PID 213 rc 0
2020.1.14 21:41:2.29: Bytes read 0 (from cmd.exe stdout)
2020.1.14 21:41:2.29: ExternProgThread has ended normally
2020.1.14 21:41:2.29: PipeinFinish
2020.1.14 21:41:2.29: App.Terminate


DosWaitChild works fine, the thread suspends correctly. The exception is sent via:
                  ULONG Exception = XCPT_SIGNAL_INTR;
                  rc = DosSendSignalException(child_PID, Exception);




Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 15, 2020, 01:10:06 am
It would appear that cmd isn't the signal focus for it's screen group, so it will not process  either the break or the intr signals. A process calls DosSetSignalExceptionFocus to become the signal focus. It looks as if cmd.exe doesn't do this, so it will only respond to the signals if it's the application that has current focus.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 15, 2020, 08:30:15 am
>A process calls DosSetSignalExceptionFocus to become the signal focus.

But I cannot execte CMD.EXE code. I tried to use DosSetSignalExceptionFocus before calling DosExecPgm, but it seems that this property of the parent process is not inherited. DosSetSignalExceptionFocus always refers to the calling process and I cannot hand over a PID in the function call.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Lars on January 15, 2020, 02:53:14 pm
I think you will have to use DosStartSession instead of DosExecPgm as you are starting a different app type.
When you do that you can also kill the session that executes CMD.EXE.
You can therefore register a handler for your process and if it fires,just kill the started session.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Lars on January 15, 2020, 02:57:00 pm
You kill a session with DosStopSession.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 15, 2020, 11:20:39 pm
I think I cannot run cmd.exe via session by using DosStartSession because only DosExecPgm inherits the standard pipes stdin/stdout/stderr. Meanwhile, I fear we're spinning circles. At the first try, I used DosStartSession and Named Pipes. And I changed to DosExecPgm and unnamed pipes with DosDupHandle with the hope of managing the Ctrl-C Interrupt communication problem. Besides this point, the two ways are basically working. But I have a weak idea that DosExecPgm is the smarter option. In both cases, I can stop it. The DosStartSession variant has got DosStopSession, the DosExecPgm variant DosKillProcess. I cannot believe that there is no solution for sending Ctrl-C to cmd.exe. If noone has got an idea to send Ctrl-C to cmd.exe, I will program a kill option in several steps: One waiting for a finished line output and the harder one with immidiately killing the process. But Ctrl-S should suspend cmd.exe. I can suspend a thread but can I suspend a process? I fear not. And what about sessions? Can I suspend sessions? How does the parent process gets informed if the session ends itseif? I also fear that it is not possible. (With the DosExecPgm variant, I can wait for finishing with DosWaitChild.) Anyway, freezing the output of the parent editor should go. But what happens if the pipe is full? Does the OS suspend the process or do I get an error?

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Rich Walsh on January 16, 2020, 09:59:58 am
I cannot believe that there is no solution for sending Ctrl-C to cmd.exe.

Of course there is.

As I understand it, you want the user of your PM-based editor to be able to press Ctrl-C or Ctrl-Break in your app to send a signal exception to cmd.exe. Your problem here is that these two keystrokes have no special meaning in PM and won't generate an exception. They're just keystrokes that show up in your message queue.

When you get one, call DosSendSignalException(pid, exception). 'PID' is the process you DosExec()'d. Exception is XCPT_SIGNAL_INTR or XCPT_SIGNAL_BREAK. I leave it to you to test whether this works.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 16, 2020, 02:55:19 pm
Laurence also assumed that it should work, but it does not.

- I start cmd.exe via DosExecPgm without failure
- I get a valid PID which corresponds to the shown value in the TOP program
- The thread where I call DosExecPgm waits correctly with DosWaitChild
- In the main thread (user input), I call DosSendSignalException with the valid PID and XCPT_SIGNAL_INTR or XCPT_SIGNAL_BREAK
- but DosSendSignalException returns error 205 and the Exception does not reach cmd.exe.

I observe the steps with a log file.

This is the problem what is discussed the last 15 posts and noone has an idea what the problem could be. A lot of reasons for malfunction are eliminated now.

Besides this point, it seems that I have got all other problems solved and the program already works fine.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 16, 2020, 05:25:20 pm
Larson Commander seems to be able to do it. The source (C++) is available at https://sourceforge.net/p/lcmd/code/HEAD/tree/ (https://sourceforge.net/p/lcmd/code/HEAD/tree/),  GPL.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 16, 2020, 09:51:19 pm
>Larson Commander seems to be able to do it.

A very interesting hint. I nearly daily use Larsen Commander. The cmd output window is indeed really interesting. It even supports marking text. I haven't checked it yet under this aspect. In opposite to my program, every command starts a new instance of cmd.exe. cmd.exe and its own PID is seen in GO and TOP. That means that it also starts DosExecPgm.
 
But - what a pity - Ctrl-C Breaks and Ctrl-Break also does not work ! Killing and exiting works. Killing ist clear, but what is exiting (while a long output is running) ? What could be the difference to killing ?This would be an option for Ctrl-C if the classic break does not work.

Anyway, searching a program which has done the job is a goodl way to find the solution.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 16, 2020, 10:05:13 pm
CTRL-C works here in Larson Manager. Test case was running a make command and pressing CTRL-C which gave a popup asking which child process to send the break to, choosing the make cmd breaks it as expected. Did crash the program when I repeated it though.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on January 16, 2020, 10:14:40 pm
Perhaps your problem is that you are sending the CTRL-C to cmd.exe instead of the child process that cmd is running, make in my example?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Lars on January 16, 2020, 11:59:36 pm
I think I cannot run cmd.exe via session by using DosStartSession because only DosExecPgm inherits the standard pipes stdin/stdout/stderr. Meanwhile, I fear we're spinning circles. At the first try, I used DosStartSession and Named Pipes. And I changed to DosExecPgm and unnamed pipes with DosDupHandle with the hope of managing the Ctrl-C Interrupt communication problem. Besides this point, the two ways are basically working. But I have a weak idea that DosExecPgm is the smarter option. In both cases, I can stop it. The DosStartSession variant has got DosStopSession, the DosExecPgm variant DosKillProcess. I cannot believe that there is no solution for sending Ctrl-C to cmd.exe. If noone has got an idea to send Ctrl-C to cmd.exe, I will program a kill option in several steps: One waiting for a finished line output and the harder one with immidiately killing the process. But Ctrl-S should suspend cmd.exe. I can suspend a thread but can I suspend a process? I fear not. And what about sessions? Can I suspend sessions? How does the parent process gets informed if the session ends itseif? I also fear that it is not possible. (With the DosExecPgm variant, I can wait for finishing with DosWaitChild.) Anyway, freezing the output of the parent editor should go. But what happens if the pipe is full? Does the OS suspend the process or do I get an error?


So why did it work for me ? I could start a session and redirect stdout and stdin without a problem.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 17, 2020, 09:39:52 am
>CTRL-C works here in Larson Manager.

When pressing Ctrl-C in Larsen Commander, I get a menu with four options:
- Ctrl-C Break
- Ctrl-Break
- exit
- kill.

Ctrl-C Break and Ctrl-Break does not work, exit and kill works. My version is 1.6 milestone 3. I run C:\>  dir /s

>So why did it work for me ? I could start a session and redirect stdout and stdin without a problem.

All works fine, but only sending the Ctrl-C exception to the cmd.exe child does not work. This is the only problem.

>Perhaps your problem is that you are sending the CTRL-C to cmd.exe instead of the child process that cmd is running, make in my example?

Yes, meanwhile I have thought about this problem, too. My main program (the editor) creates a thread where cmd.exe gets executed asynchronously, that means the thread calls  DosExecPgm call and this call returns the PID which I need for sending the exception if a user in the editor presses Ctrl-C. Then the thread waits for terminating cmd.exe by using DosWaitChild. I have only one child PID. The thread which calls DosExecPgm and then waits has only a Thread ID (TID), not a PID.

So the main interest should be at the commandline window itself. The window is a PM program and inside, there is a textmode program running. Perhaps I send the exception to the wrong part ? But I have got only one PID. (Perhaps someone can explain it further what cmd.exe is.)
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Lars on January 17, 2020, 05:20:19 pm
Rich already explained it:
your editor app is a PM app. PM apps know nothing about the special handling for Ctrl-C.
your PM app needs to treat the Ctrl-C combination in its window procedure and do a "DosSendSignalException(pid, exception)". DosExecPgm returns the PID of the program it just started (in your case:cmd.exe).
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Silvan Scherrer on January 17, 2020, 05:35:16 pm
Try to send the Signal with this snipped
http://www.edm2.com/index.php/DosSendSignalException
And see if you get the same 205.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 17, 2020, 08:15:32 pm
Silvan, a good idea. I compiled the simple text program SendException.exe.

Executing without parameter shows an help text:
sendexception.exe sends an exception to a PID. Enter the PID value in decimal first and the Execption type as second parameter.
Sample:  sendexception 197 2
INTR=1 BREAK=2 KILL=4


To recognize the cmd.exe session to observate, run go.exe in the cmd session you want to identify: GO 1.5 (https://hobbes.nmsu.edu/download/pub/os2/util/process/go_15.zip)
The PID we need is in the first row in the line of CMD.EXE, where GO.EXE is shown as child.

In this window, I run "dir /s" which takes a lot of time.
I tried the following input and I got:

Code: [Select]
[C:\]sendexception 106 1
DosSendSignalException returns 205

[C:\]sendexception 106 2
DosSendSignalException returns 209

[C:\]sendexception 106 4
DosSendSignalException returns 205

So the behaviour corresponds to my main program. Now the question is how to send these exceptions to cmd.exe. 209 means ERROR_INVALID_SIGNAL_NUMBER  what is completely suspect to me. 205 means ERROR_NO_SIGNAL_SENT, see the link Silvan posted above.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Andreas Schnellbacher on January 17, 2020, 09:01:57 pm
Sending the break signal doesn't work in EPM either. But you can send an exception to the parent thread/process instead. That stops also the child process.

I've just checked this with dir /s executed from an EPM command shell window. In plain EPM, the ProcessBreak command (Ctrl+Break) won't stop the output of CMD.EXE. That's also visible in monitoring the CPU load. In NEPMD, it's possible to kill (and restart) the process that started CMD.EXE and it stops immediately, also visible by the CPU load.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 17, 2020, 10:25:54 pm
But DosSendSignalException still returns 205.

DosSendSignalException will return 205 if it receives PID other than PID of child process.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 17, 2020, 10:41:49 pm
>In NEPMD, it's possible to kill (and restart) the process that started CMD.EXE and it stops immediately, also visible by the CPU load.

Yes, killing and restarting will be possible (it will feel quite similar). This will be only a substitute hack if the behaviour I hope for is definitely not possible.

Another possibility is to run the child synchronous and sending the signal to the parent program PID. I tried it, but now DosSendSignalException returns 1.

>DosSendSignalException will return 205 if it receives PID other than PID of child process.
But I send the signal to the child process (81 in this case).

GO.EXE output is intersting in this case:

Code: [Select]
GO! v1.2 (Rev. 93/11/27)
(c) 1993 by Carsten Wimmer

             Parent           Num. of
Process ID Process ID Session Threads Priority Name
 Dec  Hex   Dec  Hex  Dec Hex
---------- ---------- ------- ------- -------- ---->
  51   33     0    0   21  15     1     0200   CUPSD.EXE
  46   2e     0    0   20  14     1     0200   SOCKTIDY.EXE
(...)
  32   20     1    1    1   1    30     0200     PMSHELL.EXE
  80   50    32   20   24  18     4     0200       MSHELL.EXE
  81   51    80   50   24  18     1     0200         CMD.EXE

MShell is my parent program (I will change the name later). It executes its child CMD.EXE. Nevertheless, error 205 is returned.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 17, 2020, 11:08:33 pm
>DosSendSignalException will return 205 if it receives PID other than PID of child process.
But I send the signal to the child process (81 in this case).

can you give me kind of test   to run on my system ?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on January 17, 2020, 11:59:39 pm
The sample programs for the various DosException calls on edm2 all exhibit the same result.
http://www.edm2.com/index.php/DosSendSignalException (http://www.edm2.com/index.php/DosSendSignalException) Sends a signal exception to a process using it's pid. Howver sending the signal to the DosSetExceptionHandler, DosSetSignalExceptionFocus or DosAcknowledgeSignalException examples result in an error 205.
http://www.edm2.com/index.php/DosSetExceptionHandler (http://www.edm2.com/index.php/DosSetExceptionHandler)
http://www.edm2.com/index.php/DosSetSignalExceptionFocus (http://www.edm2.com/index.php/DosSetSignalExceptionFocus)
http://www.edm2.com/index.php/DosAcknowledgeSignalException (http://www.edm2.com/index.php/DosAcknowledgeSignalException)
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 18, 2020, 09:15:07 am
>can you give me kind of test to run on my system ?

See Reply #45. Here you can download SendException.exe. It is compiled with WDsibyl (Pascal). Meanwhile, I have compiled the EDM sample code with a C compiler, same result. The behaviour is the same like in my main program.

If SendSignalException and its relatives does not work at all, what is the purpose of these functions? Is it a bug? Does anyone has a Warp 3 (or 2.1) installation running?

We could send the exception to a different program to see if the problem is located in cmd.exe. I am not very familar with Exceptions. What does the Ctrl-C exception do in the target program? Is it like a keyboard input of Ctrl-C ? E.g. In ME Editor Ctrl-C is Page Down with Wordstar keyboard layout (Options - Settings - General). I haven't programmed receiving exceptions from outside yet.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 18, 2020, 09:22:56 am
>can you give me kind of test to run on my system ?
See Reply #45. Here you can download SendException.exe.

I would like to receive a copy of MShell to try to send a signal to its child - cmd.exe as shown in Reply #45.  I would try to get the answer why you received 205 in this case.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 18, 2020, 06:54:14 pm
>I would like to receive a copy of MShell

Now I have fixed some other bugs and I can show the current state of the project to others. I have changed the name now to CmdMe because MShell is already occupied (a PMShell substitute for OS/2). If anyone has got a better name for the program, please tell it here.

In the attachment, you will find the current exe (version number 0.10). I have written all important stuff into the readme text file.

If we totally fail with sending exceptions to cmd.exe, I will run cmd.exe synchronously and suspend and kill the thread. This will work, but is not the preffered solution.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 18, 2020, 08:53:12 pm
In the attachment, you will find the current exe (version number 0.10). I have written all important stuff into the readme text file.

Well, I have tested you CmdMe - everything is working as expected:

DosSendSignalException returns 205 because new processes  are _NOT_ direct children of CmdMe.   Even I use /sync key.

You can easily check this fact  with Theseus.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 19, 2020, 12:08:33 am
>because new processes  are _NOT_ direct children of CmdMe

How do I have to execute cmd.exe so it gets a direct children of my program?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 19, 2020, 12:28:08 pm
>because new processes  are _NOT_ direct children of CmdMe

How do I have to execute cmd.exe so it gets a direct children of my program?

I am not an expert - but enclosed example gives me direct child, so DosSendSignalException returns ok  - pls check.

I was not able to get direct child using DosStartSession  -  have to learn out more.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 19, 2020, 05:33:49 pm
I think that DosStartSession is the wrong way because Sessions can include several processes and we need to get the two processes as close as possible.

It tried your sample programs and they indeed work.
The differences to my program are:

- They are running in one single cmd window. I have got a PM app (the parent editor) and a child (cmd.exe) which is not a PM program.
- I am starting DosExecPgm from a thread, your sample program has no theads.

I will now modify your parent send program to a PM app and I will see what happen.
Then I will try to execute the child from the main thread in my program.

Hope we find the reason why your program works and mine not.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Wim Brul on January 19, 2020, 07:47:13 pm
Hi Martin,

I think that DosStartSession is the wrong way because Sessions can include several processes and we need to get the two processes as close as possible.

I don' think so. Using DosStartSession() is the only way to go:

You must use DosStartSession() to start your non-PM process from your PM process.

Within your non-PM process you may then use DosExecPgm() to invoke the Command Processor.   

Your non-PM process may then use DosSendSignalException() to that Command Processor process.

You will have to find a way to signal this request from your PM process to your non-PM process.

One could think of using a system semaphore like e.g. \SEM32\STOPTHAT to do that.

Wim Brul
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 19, 2020, 11:17:05 pm
Wim,
that sounds very plausible. Your description means, exceptions can only be sent from a non-PM process to a non-PM process?

If I understand the DosSendSignalException docu right, the relationship between the processes (child or not) does not matter.

Quote
DosSendSignalException sends either an XCPT_SIGNAL_INTR (Ctrl+C) or an XCPT_SIGNAL_BREAK (Ctrl+Break) signal exception to another process.
But I tried to run os4users parent and child exe from different sessions and the post does not reach the "child" anymore. So it indeed seems that a relationship is necessary.

So I could start two non-pm programs, the cmd.exe and a text-based helper exe which simply sends the message forward to cmd.exe.

Meanwhile, I have tried that both non-pm programs are childs of my PM program and are started via DosExecPgm (asynchronous to get the PID). The parent can tell the helper the PID of the brother/sister. This doesn't work, too. That means, the editor has to execute the helper and the helper has to execute cmd.exe. So my editor gets a grandparent. I'll try this and report here.

Wim, why do you think that I need to implement a new session? Cmd.exe currently runs in the same session as the parent editor.

What is a good tutorial for semaphores with OS/2 ? In my german OS/2 programmers book, semaphores are missing.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Wim Brul on January 21, 2020, 12:23:00 pm
Quote
Your description means, exceptions can only be sent from a non-PM process to a non-PM process?

Establishing an exception handler for Ctrl+C and Ctrl+Break is only supported for vio-window and full-screen sessions.
The pmshell session exception handler has its own way to deal with exceptions and DosSendSignalException has no effect.
See Figure1 - Proposed for working implementation. Shows the barrier between pmshell session and screen group session.

Quote
If I understand the DosSendSignalException docu right, the relationship between the processes (child or not) does not matter.

Exceptions are handled within a screen group on a Last-In-First-Out basis. Therefore a process within a screen group can always establish its own exception handler, which it might need, whether or not it is the parent, child or grandchild etc. within that screen group session.     

Quote
So I could start two non-pm programs, the cmd.exe and a text-based helper exe which simply sends the message forward to cmd.exe.

When you start two non-pm programs from you pm program then your helper process runs isolated from the command processor process.
See Figure2 - Proposed NOT working implementation. Shows barrier between pmshell session and both screen group sessions.

Quote
Wim, why do you think that I need to implement a new session? Cmd.exe currently runs in the same session as the parent editor.

I don't know how to explain it better. It is because DosSendSignalException() used by your PM program cannot affect cmd.exe then.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 21, 2020, 03:26:27 pm
Thank you Wim for the further explanations and figures. So the "grandfather" concept should work:

Grandfather (my PM editor) starts parent (cmdhelper.exe commandline program) via DosExecPgm - synchronous or asynchronous, with own session or not makes no difference at this point, because PID is not needed. I can send basic information from the grandparent to the parent when starting the parent via commandline parameters (name of the semaphore, cmd.exe or 4os2.exe to start). (The name of the semaphore need to have a counter because perhaps several instances of my program are running, or I use WinQueryWindowProcess and use the grandfather PID for a unique semaphore name.)

The parent executes the child (cmd.exe) via DosExecPgm in asynchronous mode and remembers the PID of the child.

If a user presses Ctrl-C in the grandfather PM program, the parent cmdhelper.exe program gets informed via Semaphores.  Then the parent sends the exception to the child via DosSendSignalException.

GO.EXE will show the structure:
  cmdme.exe (PM editor)
       cmdhelper.exe
               cmd.exe

Hope this concept is OK now ? (I can start the parent in a new session, no problem, but don't understand it, because if I use the same session as in the PM program, both cmdline programs are still in the same session.)
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Wim Brul on January 21, 2020, 06:25:18 pm
Quote
Grandfather (my PM editor) starts parent (cmdhelper.exe commandline program) via DosExecPgm - synchronous or asynchronous, with own session or not makes no difference at this point, because PID is not needed.

The use of DosExecPgm() is incorrect at this point. DosStartSession() must be used to start the child session with different SSF_TYPE_WINDOWABLEVIO session type.

Quote
Hope this concept is OK now ? (I can start the parent in a new session, no problem, but don't understand it, because if I use the same session as in the PM program, both cmdline programs are still in the same session.)

Almost correct. Have a close look at Figure1 again. You will notice DSS written vertically to indicate that the editor uses DosStartSession() to start the helper. You need to use SSF_RELATED_CHILD to establish a parent session/child session relationship. A parent process/child process relationship is not established.

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 21, 2020, 08:50:57 pm
Thank you, especially Wim, Laurence and OS4USER for your comments. Now I think I will manage it without help.

Programming is like real life. I send signals to my childs, and they did not receive the messages... Make your homework now, come to dinner, go to sleep, and no response...
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: OS4User on January 21, 2020, 10:12:42 pm
Thank you, especially Wim, Laurence and OS4USER for your comments. Now I think I will manage it without help.

Martin, I have to admit that I tested your cmdme.exe  in a wrong way  -  I started new cmd  with  "start cmd" command from command line provided by cmdme.exe  and a new process was not a direct child of cmdme  -  that is what I stated.

Now I traced this case  with proper scope (I hope :) ) and I can see  that cmdme.exe  is delivering new process "cmd"  automatically after start. Yes,  this "cmd"  is a direct child of cmdme.exe.

If I press Cntr-C  in cmdme.exe  window  it generates DosSendSignalException with a proper PID and signal.

DosSendSignalException returns you ERROR_NO_SIGNAL_SENT because cmd.exe does not report itself as "receiver of signals".

From  an example  I posted earlier   you may see   how  a potential child ("DosSetExceptionHandler.exe")  is organized to be a  "receiver of signals".

Please recheck it   - but it looks like  it is working in this way.

Regards
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 21, 2020, 10:52:20 pm
> you may see  how  a potential child ("DosSetExceptionHandler.exe")  is organized to be a  "receiver of signals"

Even if I can send an exception from the PM parent to a cmdline child, the child has to be prepared. So a helper program would be necessary nevertheless. I think I will program Wims concept, I am tired of trials... I cannot re-organize cmd.exe. Wims concept is not really compilicated. I have only to lern semaphores. The helper program will be only a few lines of code.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Lars on January 21, 2020, 11:23:40 pm
Why do you need an intermediate process ? Why don't you just use "DosStartSession" in your PM app to run cmd.exe, catch the Ctrl-C combination in your PM app (just look for that Ctrl-C key combination in your window procedure) and then do a "DosStopSession" in your PM app. That should work as long as you started cmd.exe as RELATED.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 22, 2020, 08:40:50 am
Yes, killing and restarting cmd.exe would work, too. I could also start with DosExecPgm and kill the process with the PID. But it is not a gentleman solution. Files could rest open (when for example copying files). The sub processes/programs cmd.exe would also get killed or can hang. Sending Ctrl-C to cmd.exe and a correct interrupt is strongly recommended. Cmd.exe always finishes a file copy when pressing Ctrl-C, for example. My program will have one limitation I cannot omit (that direct Screen IO buffer output is not supported), and this should really be the only limitation. Wim and OS4user sample programs showed us that it will be possible with a helper program of a few lines code which will have a size of 30 kB.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 23, 2020, 03:53:58 pm
Meanwhile, I found the programming tutorial and description of semaphores:
IBM OS/2 Warp Development Manuals (http://www.edm2.com/index.php/List_of_OS/2_Documentation), especially the document cp3.inf.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 28, 2020, 11:07:40 pm
I have written now the communication between the PM editor and the helper program via unnamed semaphores and shared memory. When the PM editor runs the helper, the command parameters hold the pointer values of the shared memory and the semaphores. This all works fine already.

I tried to run the program in the grandfather - parent - child variant and it did fail - Wim was absolutely right with his forecast, when he said, I need a separate session for the helper and the cmd.exe program. The parent cannot send exeptions to the child if it has been executed as a process from another program (grandfather) - I indeed get an error when calling DosSendException to cmd.exe in the helper program.

Because the editor has to start the helper program in a separate session (DosStartSession), the inheritance of stdin, stdout und stderr via DosExecPgm is not available. So I do not use DosDupHandle in the PM editor. Instead, I use DosCreatePipe and inform the helper program about the file handle values of the three pipes via commandline parameters.

But now the helper program has to set stdin, stdout and stderror to the values which are received from the PM editor via commandline parameters. In Windows, there's a function SetStdHandle, and I cannot use DosDupFile, because DosDupFile (parampipehandle, &STDIN_HANDLE) does not work of course (because STDIN_HANDLE has the fix value 0). How can I set stdin, stdout and stderr to my specific handle values before executing cmd.exe and inheriting stdin, stdout and stderr? I am sure that the solution is simple, but I have no idea anyway.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on January 29, 2020, 10:08:31 pm
I have found the solution in the sample code popen.c (https://fossies.org/dox/cvs-1.11.23/popen_8c_source.html) again:

Code: [Select]
   #define STDIN 0
    HFILE Tmp;
    Tmp = STDIN;
    DosDupHandle (stdin_from_editor, &Tmp);

and similiar with stdout and stderr. It seems that DosDupHandle handles the values 0, 1 and 2 different from a memory pointer. But it is necessary to use a Tmp file handle variable.

But it does not work yet. It seems that unnamed pipes are only allowed between processes in one single session. This is not documentated, but in the documentation I found: "Unnamed pipes are useful to transfer data between related processes".

So it seems that I need to switch to named pipes again. To preclude a programming failure, does someone know it definitely ?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Wim Brul on January 30, 2020, 07:18:25 pm
Quote
So it seems that I need to switch to named pipes again. To preclude a programming failure, does someone know it definitely ?

With DosStartSession() a parent process/child process relationship is not established. Hence named pipes must be used between editor and command processor.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 04, 2020, 10:39:34 am
Meanwhile, the new concept of the cmdme editor with a cmdhelper exe works! Thanks to Wim especially. So I have managed all the programming close to OS/2. Ctrl-C and Ctrl-Break (I see no difference) works now. Ctrl-S pause is implemented directly in the editor (cmd.exe runs straight on until the pipe buffer is full), because I assume that I can only suspend threads but not sessions.

There are some additional questions left for further programming:

(1) It would be interesting to see the communication between cmd.exe and the PM window where cmd.exe is executed. Is there any way to read/redirect this information? (I have no idea what to do with this information yet, but it could be useful.)

(2) If a user executes a crt VIO program (like DFSEE) which does not support stdout and directly writes into the screen buffer, can I recognize these programs by examining the binary? So there are three different types of OS/2 programs: PM, commandline with standard IO, commandline which writes directly to the screen buffer. Then I would turn the hidden cmdhelper/cmd.exe window to front when exeuting the last program type.

(3) I have found an old OS/2 1.X program which simply shows a colored directory tree (see attachment). This program does not generate ANSI ESC sequences, but has normal stdout. In cmdme, it is shown without colors. I do not understand how it can show colors, support stdout and does not generate ANSI ESC sequences.

(4) Where do I find a description of cmd.exe keyboard input? (E.g. cursor up restores old input, Ctrl-C breaks and so on). "help cmd" shows only the commands you can type.

(5) When starting cmd.exe from helper exe, it is always shown:
Code: [Select]
SYS1034: The system cannot find the command processor in the path specified.
OS/2 Command Interpreter version 4.5
But cmd.exe runs and all works fine. It does not matter if I start cmd.exe with or without full path name. 4os2 writes 'Unknown command "Program" ' instead and also works.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 04, 2020, 02:44:01 pm
Quote
) It would be interesting to see the communication between cmd.exe and the PM window where cmd.exe is executed. Is there any way to read/redirect this information? (I have no idea what to do with this information yet, but it could be useful.)
As you are reading and writing texts strings to/from the pipe do an additional write of the same strings to a log file after reading/writing them to/from the pipe is probably the simplest method.
Quote
(3) I have found an old OS/2 1.X program which simply shows a colored directory tree (see attachment). This program does not generate ANSI ESC sequences, but has normal stdout. In cmdme, it is shown without colors. I do not understand how it can show colors, support stdout and does not generate ANSI ESC sequences.
It's probably mixing standard i/o and vio calls. and overlaying one on top of the other. So you would get colour output on a terminal and plain text if the output was redirected.

Quote
(4) Where do I find a description of cmd.exe keyboard input? (E.g. cursor up restores old input, Ctrl-C breaks and so on). "help cmd" shows only the commands you can type.
The KEYS command implements command line editing.
Quote
SYS1034: The system cannot find the command processor in the path specified.
OS/2 Command Interpreter version 4.5
The first of the ASCIIZ strings passed as an argument to DosExecPgm should be the name of the command interpreter, "cmd\0" of "40s2\0"  for the os2 command intepreter and 4OS2 respectively. Otherwise it treats the first command you pass as the name of the program to load, which, if it an internal command or an option paramter (DIR or /K for example) you'll get a command not found error

Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 04, 2020, 03:04:42 pm
Here's a hello world example of overlaying vio and std i/o
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 04, 2020, 03:06:01 pm
Laurence,

(5) works now. I have indeed to pass the argument "cmd.exe /K\0" instead of "/K". The error message has vanished now. Does that mean that two instances of cmd.exe are executed?

(4) "help KEYS" fits my purpose, thank you.

(1) Of course I can read stdout, and this works fine already. My query relates to the other stuff which is sent to the PM window like changing the size with "mode" or deleting the screen buffer with "cls" and so on.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 04, 2020, 03:39:11 pm
Quote
(5) works now. I have indeed to pass the argument "cmd\0 /K" instead of "/K". The error message has vanished now. Does that mean that two instances of cmd.exe are executed?
No there's only one instance of the command shell,  It's a convention in C that the first entry in the array of command parmaters passed to a program is the name of the command. argv[0]. The called program skips the first parameter if it's the name of the called and reads the rest of the paramters. If the first parameter isn't the name of the shell then it treats it as an external command. If there are no parameters to pass then you can pass a null string and you won't get an error.

Quote
Of course I can read stdout, and this works fine already. My query relates to the other stuff which is sent to the PM window like changing the size with "mode" or deleting the screen buffer with "cls" and so on.
Write the commands you are sending to stdin to a log file in addition to sending them to stdin. Or are you asking how to capture the return code from the command you are sending to stdin? In which case check the %ERRORLEVEL% value immediately after issuing the command.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 05, 2020, 10:22:51 am
I want to capture the VIO calls cmd.exe sends to the PM window or to the screen (fullscreen) when e.g. a user types "mode" or "cls".
The problem I have is that I see the user input in the editor of course and the editor can interpretate these command by itself, but when cmd.exe executes a cmd file which holds "mode" or "cls", it won't get interpretated by the editor.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 05, 2020, 12:21:16 pm
Quote
I  want to capture the VIO calls cmd.exe sends to the PM window or to the screen (fullscreen) when e.g. a user types "mode" or "cls".
To capture the indivdual calls you'd need to enable debugging in the call to DosExecPgm by setting the execFlag to EXEC_TRACE and using DosDebug to trace the execution of the process. I'm pretty sure this is not what you want to do, It dependson the debugging symbols being available for the executable and/or the ability to identify the assembly langage calls to the vio functions.

Quote
The problem I have is that I see the user input in the editor of course and the editor can interpretate these command by itself, but when cmd.exe executes a cmd file which holds "mode" or "cls", it won't get interpretated by the editor.
This is simpler, check for a .cmd extension then load the cmd file into a buffer and interpret it line by line and once you've done that send the cmd file to the cmd process. Implementing a full cmd file interpreter is non trivial,  but if you are only interested in specifc commands then it can be simplified to a straight word search.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 05, 2020, 05:56:39 pm
Quote
This is simpler, check for a .cmd extension then load the cmd file into a buffer and interpret it line by line

Yes. Then I have only to interpretate my commands like mode and cls and I have to interpretate call to open cmd files recursive. That's not difficult to program. But I will introduce the rule that .cmd has to be added, otherwise cmd.exe executes the cmd file directly. (The user has the choice. It would be possible, but difficult to check whether a command is a cmd file or not.)

But this concept will fail with REXX scripts. I tried to copy a REXX script into the clipboard, then I copied it to the cmd.exe window, and it fails. So REXX scripts have to be executed directly by cmd.exe. If the user tries to type myrexxscript.cmd, then cmd.exe has to exeute the file directly nevertheless. REXX programs awas begin with /*, so they are easy to identify.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Dave Yeo on February 05, 2020, 06:30:41 pm
Doesn't cmd.exe just pass execution to the REXX interpreter? You could pass REXX to pmrexx.exe and execute it that way perhaps.
There's also the EXTPROC keyword which can pass execution to another interpreter, I've used it to call sh scripts and it's used for Perl scripts, eg
Code: [Select]
extproc perl -S
#!L:/Perl/bin/perl
    eval 'exec L:/Perl/bin/perl -S $0 ${1+"$@"}'
        if $running_under_some_shell;
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 05, 2020, 07:18:30 pm
For a rexx script check for the two charactee /* as the first two characters in the cmd file. You can extend this to other intepreted scripts by checking whether extrpoc or #!(4OS2 extension if the UnixPaths option is enabled) are the first characters in athe command fiile. and that is followed by the name of the external processor, perl, python, etc.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 15, 2020, 12:47:06 pm
- Problem with named pipes error 231 "pipe busy" when DosOpen
- Problem with cmd.exe idle state (program running or waiting for user input with prompt)

I thought that all OS-specific programming works fine now, but I still have got a problem with named pipes. When the helper program opens the named pipe, I sometimes get an error 231 (pipe busy). I cannot reproduce the problem regulary. It even occurs if I run only one instance of my program. I checked all named pipe API call returns. All works fine up to the DosOpen call. DosDisConnectNPipe returns 0 when exiting the last program session and DosCreateNPipe in the current session also returns 0. DosClose in the helper program of the previous session also returns 0. The problem does not occur when running the first session of the program after booting the computer. So it seems that the previous cmdme session did not clean up all properly. Any ideas?

Then I have got another problem. I have realized that the editor has to know whether cmd.exe is in idle state (waiting for user input) or not. Because if cmd.exe is exeuting a program and e.g. the user has to enter "Y" or "N", for example, a user input has to be sent immideately to cmd.exe. If there's a prompt and the user wants to enter a new command, the finished complete line of the new command with an ending Return character is sent and before, the user can change the input, has tab completion support and so on. I can read the prompt, but the prompt can change with the PROMPT command. A program which is executed by cmd.exe could generate a similar input. Any ideas how to query this state is appreciated. (I have got the session ID of the helper which executes cmd.exe. Perhaps I can query this state by querying the current session? Or I need a very intelligent interpretation of the proposed prompt string?)
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 15, 2020, 02:06:09 pm
Quote
- Problem with named pipes error 231 "pipe busy" when DosOpen
Generally you should check for this error and issue a DosWaitNPipe on reciept of this error followed by another call to DosOpen.
Quote
DosClose in the helper program of the previous session also returns 0
Are you issuing a DosClose call on the pipe from the server application?
Quote
Named-Pipe Considerations

DosClose closes a named pipe by handle. When all handles that refer to one end of a pipe are closed, the pipe is considered broken.

If the client end closes, no other process can reopen the pipe until the serving end issues DosDisConnectNPipe, followed by DosConnectNPipe.

If the server end closes when the pipe is already broken, the pipe is immediately deallocated; otherwise, it is not deallocated until the last client handle is closed.

The pipe isn't deallocated until both server and client handles are closed.

Quote
Or I need a very intelligent interpretation of the proposed prompt string?)
Isn't the user seeing the contents of stdout?
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 15, 2020, 02:57:27 pm
Quote
Are you issuing a DosClose call on the pipe from the server application?
The server app (the PM editor) calls:
DosCreateNPipe
DosConnectNPipe (waits for DosOpen on client)

Meanwhile, the client (the helper which executes cmd.exe) calls DosOpen

Then the server continues. I do not use DosOpen in the server.

When the program gets finished, the client runs DosClose first. Then the server calls DosDisconnectNPipe.

I assumed that in the server I only have to use DosCreateNPipe, then Connect and Disconnect which is a substutute for DosOpen and DosClose. So do I have to use both DosOpen and DosClose in the server app or only DosClose?

------------
Quote
Isn't the user seeing the contents of stdout?
Yes, the user sees stdout in the PM editor immediately. But the PM editor has to know whether to send each keyboard user input character to cmd.exe immediately (within a sub program) or (prompting/idle) to wait until the user presses Enter.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Andreas Schnellbacher on February 15, 2020, 04:18:36 pm
I have realized that the editor has to know whether cmd.exe is in idle state (waiting for user input) or not.
You can parse the last line for a usual prompt. That's not perfect, but it works suitably.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 15, 2020, 04:30:26 pm
You only have to use DosClose in the server because threads in the process that creates the pipe have access to it and don't need to call DosOpen. DosDisconnectNPipe doesn't deallocate the pipe, which is why it can be reopened with another call to DosConnectNPipe without haveing to call DosCreateNPipe to recreate it. .
Quote
DosDisConnectNPipe is issued by a server process to acknowledge that a client process has closed a named pipe. (If a client process tries to issue DosDisConnectNPipe, ERROR_BAD_PIPE is returned.) The pipe cannot be opened by another client process until the server process issues this function, followed by DosConnectNPipe.

Quote
But the PM editor has to know whether to send each keyboard user input character to cmd.exe immediately (within a sub program) or (prompting/idle) to wait until the user presses Enter.
If it's using standard i/o then it doesn't matter whether the editor waits for the enter key to be pressed before sending the character, If it's doing single character i/o and you send it a line of characters then they'll be buffered for subsequent reads, if it's only expecting a single character input then it should ignore the enter key, and if it's not flushing the input buffer before each read then that's a bug. If it's using Vio calls to read character strings then you can't use it from a pipe.

So, if you've got a prompt Y/N and you send "Y\n" the newline "\n"  should be discarded by the calling program.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 15, 2020, 04:36:52 pm
In regard to parsing the prompt the simplest solution is to set the prompt to a known state at the start of the session and prevent the user from changing it during the session.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 15, 2020, 09:18:54 pm
If I understand it right, DosConnectPipe and DosDisConnectPipe using in the server only waits for client action opening and closing the pipe on the client end. I have to DosClose the named pipe both on the server and on the client end.

When ending the program, I inserted DosClose after DosDisConnectNPipe in my server program. And, alternatively, I use DosClose in the server program instead of DosDisconncectNPipe which should be not necessary (because I do not want to open the pipe again).

But the casually pipe busy error after restarting the program and its client still remains. ???
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Laurence Pithie on February 16, 2020, 01:29:21 am
My apologies, I didn't read your orignal post properly. The problem occurs when the client program calls DosOpen. It sounds like a race condition, The server creates the pipe and then calls DosConnectNPipe to put it into a listening state. If the client program calls DosOpen before the DosConnectNPipe call completes the client will get an ERROR_PIPE_BUSY returned from the call to DosOpen. You can either check for the error and issue a DosWaitNPipe call or synchronose the pipe setup in the server with the opening of the pipe in the client using a semaphore.
Title: Re: problems programming a new text window for cmd.exe/4os2.exe
Post by: Martin Vieregg on February 16, 2020, 09:12:42 pm
Yes, Laurence, you are right. It is a race between the PM editor and the helper program. DosWaitNPipe in the helper program fits the purpose:

Code: [Select]
#define logwrite if (logging) {logf = fopen (ExeDir, "at"); fprintf
#define logend; fclose (logf);}
FILE *logf;
char Inpipename[50];
    logwrite (logf, "trying opening Inpipe %s\n", Inpipename); logend;
    rc = DosOpen (Inpipename, &cmd_stdin, &ulAction, 0, FILE_NORMAL, FILE_OPEN, OPEN_ACCESS_READONLY |
                   OPEN_SHARE_DENYNONE,(PEAOP) NULL);
    if (rc != 0) {logwrite (logf, "error %i Open Named Pipe %s\n", rc, Inpipename); logend;}
    if (rc == 231) { //Pipe is busy because Server did not finished calling DosConnectNPipe yet
      Timeout = 1000;/*ms*/
      rc = DosWaitNPipe(Inpipename, Timeout);
      logwrite (logf, "DosWaitNPipe returns %i\n", rc); logend;
      //try opening pipe again
      rc = DosOpen (Inpipename, &cmd_stdin, &ulAction, 0, FILE_NORMAL, FILE_OPEN, OPEN_ACCESS_READONLY |
        OPEN_SHARE_DENYNONE,(PEAOP) NULL);
      logwrite (logf, "second trial Open Named Pipe %s returns %i\n", Inpipename, rc); logend;
    }
    else {logwrite (logf, "successful open Named Pipe %s\n", Inpipename); logend;}

I can see in the logfile, that sometimes the additional code gets executed. Now I have to implement much more easier stuff.