Author Topic: MAKEFILES suck!!!  (Read 5709 times)

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
MAKEFILES suck!!!
« on: November 25, 2020, 04:51:23 am »
...alright, maybe not that badly, I have used them for much smaller projects in the past and they were great, but I'm attempting to compile the PUMonitor utility here and that provided makefile doesn't work with my VAC 3.6.5 install.

So help me out please, my eyeballs are red from reading about NMAKE32 (as opposed to NMAKE), but clearly I'm still confused...LOL!  :-\

Here is what I have:

1) structure of the project source files
Code: [Select]
Directory of G:\code\source\os2\pumonitor\src

11-24-20  9:25a         <DIR>      0 ----  .
 8-25-18  5:42p         <DIR>    369 ----  ..
 2-19-02  1:13p           302      0 a---  build.opt
 8-25-18  5:42p         <DIR>      0 ----  include
 2-19-02  1:13p           105      0 a---  library.rsp
11-23-20 11:50p         2,151     35 a---  makefile
 8-25-18  5:42p         <DIR>      0 ----  obj
11-24-20 10:24p         2,330     35 a---  pumonitor.mak
 8-25-18  5:42p         <DIR>      0 ----  source

Basically the *.c and *.cpp is all in source, *.h and *.hpp in include, and the remaining *.ico, *.rc and *.lib files are in source.

2) 'makefile' is the original makefile distributed with the project source, but despite the fact that it reads like it's meant to work with VAC, it doesn't work with my 3.6.5 setup here

3) 'pumonitor.mak' is my attempt at converting the project sources and the build process as I understand it to be structured in the original 'makefile' into a VAC 3.6.5 version

Sooo...having said that, when attempting to process with NMAKE32 I get the following error:

Code: [Select]
MAK3035: Do not know how to make target 'cell.cpp'.

Now this part is really confusing for me, because if I force the full path for the source files in the target, all works fine, here is the section we are talking about:

Code: [Select]
cell.obj    : cell.cpp .\include\cell.h .\include\cvars.h .\include\util.h

...substituting with

Code: [Select]
cell.obj    : .\source\cell.cpp .\include\cell.h .\include\cvars.h .\include\util.h

...now allows me to pass that test but still fails with the following:

Code: [Select]
MAK3035: Do not know how to make target 'cell.obj'.

...which of course can be addressed if I provide an explicit command, such as:
Code: [Select]
$(CC) /c /Fo$@  $<

...then compiles, although it still producs a pile of errors:

Code: [Select]
...
icc.exe /c /Focell.obj .\source\cell.cpp
IBM* C and C++ Compilers for OS/2*, AIX* and for Windows NT**, Version 3.6
(C) Copyright IBM Corp. 1991, 1997   All Rights Reserved.
* registered trademarks of IBM Corp., ** registered trademark of Microsoft Corp.

.\source\cell.cpp(19:10) : error EDC3008: Source file <cell.h> cannot be opened.
.\source\cell.cpp(20:10) : error EDC3008: Source file <cvars.h> cannot be opened.
...

All of this begs the question: why isn't my Inference Rule working? It clearly spells out where to find all the *.h and *.hpp files....so what gives?

Here is the Inference Rule section in my makefile:

Code: [Select]
# The Make utility looks in the directory specified by frompath for files with the fromext extension.
# It executes the commands to build files with the toext extension in the directory specified by topath.
# {frompath}.fromext{topath}.toext
# commands
# :

{.\source}.c{.\obj}.obj:
    $(CC) /c /Fo$@  $<
   
{.\source}.cpp{.\obj}.obj:
    $(CC) /c /Fo$@  $<
   
{.\source}.lib{.\obj}.obj:
    $(CC) /c /Fo$@  $<
   
{.\source}.rc{.\obj}.res:
    $(RC) /r $< $@
   
{.\include}.h{.\obj}.obj:
    $(CC) /c /Fo$@  $<

{.\include}.hpp{.\obj}.obj:
    $(CC) /c /Fo$@  $<

This is far beyond anything I've tried before...so obviously I'm lost!!! lol

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: MAKEFILES suck!!!
« Reply #1 on: November 25, 2020, 08:21:27 am »
Did you read the readme.txt? To quote from v2-1b166,
Quote
To build PU Monitor from sources, you'll need Watcom C++ 10.6 and
MAKE utility from Borland C++ 2.0 for OS/2 (it is also provided with TASM 4.1).
There are no known dependencies on this tools so you may try to compile
sources by another compiler. All you'll need is to write new MAKEFILE.

I'll attach the BC make I have. You'll have to install OW if you don't already have it, or write a nmake makefile.

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #2 on: November 25, 2020, 08:51:11 am »
if the target of your inference rule is
{.\obj}.obj
then a concrete target would be
{.\obj}\cell.obj
Because this is clumsy,many makefiles define Makros for Source and Object directories.
The same holds for dependencies.
As to included h files, apart from being a dependency, they also need to be found by the compiler. That is, you will need to add the -I switch with the proper subdirectory to your compile inference rules.

Also, the inference rules dependent on h files will not work as you intend. That's because they will attempt to compile the h file ( Makros $< is the input file) and NOT the c file that includes them which is likely your intention).
« Last Edit: November 25, 2020, 09:12:30 am by Lars »

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #3 on: November 25, 2020, 09:13:32 am »
if the target of your inference rule is
{.\obj}.obj
then a concrete target would be
{.\obj}\cell.obj
Because this is clumsy,many makefiles define Makros for Source and Object directories.
The same holds for dependencies.
As to included h files, apart from being a dependency, they also need to be found by the compiler. That is, you will need to add the -I switch with the proper subdirectory to your compile inference rules.

Also, the inference rules dependent on h files will not work as you intend. That's because they will attempt to compile the h file ( Makros $< is the input file) and NOT the c file that includes them which is likely your intention.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #4 on: November 25, 2020, 05:10:26 pm »
Hi Dave!

Did you read the readme.txt? To quote from v2-1b166,
Quote
To build PU Monitor from sources, you'll need Watcom C++ 10.6 and
MAKE utility from Borland C++ 2.0 for OS/2 (it is also provided with TASM 4.1).
There are no known dependencies on this tools so you may try to compile
sources by another compiler. All you'll need is to write new MAKEFILE.

I'll attach the BC make I have. You'll have to install OW if you don't already have it, or write a nmake makefile.

Oh yes sure read that file, c'mon...give me a little creadit here  :(. However, excellent question nonetheless, so let me explain: given that 2nd last sentence re: "...may try to compile sources by another compiler..." made me think that I should try my VACPP 3.6.5 install fist - since this is what I normally experiment with. I honestly thought it should be nearly trivial to convert the existing makefile to something that works with VACPP. Not so, so far anyways.

But...since I have BCOS2 installed as well, I took your advice and re-pointed to the BC make.exe as opposed to trying to re-create a native VACPP one, while still using VACPP for the compile and linkage duaties. Indeed sir...excellent idea to re-point the obvious, that succeeded!!!

I was able to compile the app, which then ran successfully. Further on, I made the changes to the code sections which I believe were causing a problem and whala...happy times are here again, PUMonitor works fine now w/o causing the annoying CPU spikes. I extended the existing limit of 4 CPUs out to 6 CPUs (that's applicable to both the global variable that controls the max CPU limit as well as the underlying structs that support the stats tracking for these).

The un-expected ROI is that fact that enabling the Hi-Res Timer for the 'Memory Meter' function no longer causes my uptime counts to get all whacky!!!

Again, thank you for that suggestion.


Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #5 on: November 25, 2020, 05:18:02 pm »
Hi Lars,

if the target of your inference rule is
{.\obj}.obj
then a concrete target would be
{.\obj}\cell.obj
Because this is clumsy,many makefiles define Makros for Source and Object directories.

Yeah, I think I understand that now. I also suspect that the Inference Rules I created cover far too large of a grouping of file extensions, which is why that additional comment you provided below makes that much more sense to me now.

...The same holds for dependencies. As to included h files, apart from being a dependency, they also need to be found by the compiler. That is, you will need to add the -I switch with the proper subdirectory to your compile inference rules....

Well, but you see this already in place because I do have the CC and CFLAGS macros defined and specifically the CFLAGS macro shows the following:

Code: [Select]
CCFLAGS     = -O+ -Oc+ -Ss+ -G5 -Q+ -Gm+ -I..\include

However, what I think is hapening is that I do not end up compiling the individual objects with those flags, instead I use a straight forward '$(CC) /c /Fo$@  $<' and that misses the mark.

So I'll re-define a few more things in my VACPP makefile, I'd like to make this work, if anything it's a great learning experience.

Appreciate the feedback Lars!

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #6 on: November 26, 2020, 12:10:10 am »
I was able to compile the app, which then ran successfully. Further on, I made the changes to the code sections which I believe were causing a problem and whala...happy times are here again, PUMonitor works fine now w/o causing the annoying CPU spikes. I extended the existing limit of 4 CPUs out to 6 CPUs (that's applicable to both the global variable that controls the max CPU limit as well as the underlying structs that support the stats tracking for these).

The un-expected ROI is that fact that enabling the Hi-Res Timer for the 'Memory Meter' function no longer causes my uptime counts to get all whacky!!!

Again, thank you for that suggestion.

Why don't you dynamically query the number of processors (DosQuerySysInfo with QSV_NUMPROCESSORS) and then allocate the structures dynamically ?
I have a system with 8 cores. I'll then miss all the good things. If that is too complicated, then use 64 entries. That is the absolute maximum that the SMP kernel can handle. Dynamically is preferred, though.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #7 on: November 26, 2020, 05:10:54 pm »
Hi Lars,

Why don't you dynamically query the number of processors (DosQuerySysInfo with QSV_NUMPROCESSORS) and then allocate the structures dynamically ?
I have a system with 8 cores. I'll then miss all the good things. If that is too complicated, then use 64 entries. That is the absolute maximum that the SMP kernel can handle. Dynamically is preferred, though.

Well, truth be told this CPU-spike issue really belonged in the nuisance category for me. The code itself is fairly complex in a sense that the author developed his own display-cell library (which does appear to be pretty feature-rich), but that also requires you to trully understand how the STATs tracking converts into the display-cell updates, and that is beyond my current level of "wanting" to get it done.

I think you are absolutely right that making this fully dynamic would be the best approach, which is really what I didn't get about the code to start off with as I attempted to understand how the information DosQuerySysInfo provides is being used. There appears to be a real gap here in a sense that this API call is used to determine how many CPUs exist, but beyond that the program logic itself seems to be hard-coded for a max of 4 CPUs.

Given that my current goal has been accomplished I will expand my static fix up to 12 CPUs, which today gets anyone up to that Ryzen 9 5900X level (12 core/24 thread). One of these days, when it is viable to run something like the Threadripper maybe a re-write of this code would be worthwhile, eh?

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: MAKEFILES suck!!!
« Reply #8 on: November 26, 2020, 06:30:17 pm »
Use 64 instead of 12 and you will be good forever. OS/2 will never ever support more than that. THAT is effectively hardcoded in the number of GDT entries reserved for "per-core" data structures.