Author Topic: Closed source parts of eComStation  (Read 55200 times)

Fr4nk

  • Guest
Re: Closed source parts of eComStation
« Reply #15 on: January 15, 2014, 10:01:07 am »
Thanks Doug this is exactly what I think.
To demand is always easy - to fulfill is not. We should eat humble pie and, be realistic and focus on the things we really need in this moment.

For things like several virtual machines on one computer there are much better platforms than os2, with better hardware support, better supported virtualization software and better memory management. And as this topic is a moving target eCS will never compete with these systems. We are a handful of people. Most of the new ported applications are completely untested. The eCS core features are reliability, WPS, Rexx, special and unique programs, small requirements. Thats it what I would try to support and to expose.

You can say 4GB limit is a bug, you can also says it is great that eCS needs only 2 GB. Its a feature. It is a damn good thing that eCS don't wastes your RAM. It really good that we don't have this helix with more hardware requirements with each version and reciprocal upgrades in hard and software (news OS needs new hardware, new hardware needs new os, endless loop while we doing still the same with our computers without any significant changes) we hate on Windows so much.

Btw im writing this on a computer with 512 MB RAM
« Last Edit: January 15, 2014, 10:19:33 am by Fr4nk »

dbanet

  • Guest
Re: Closed source parts of eComStation
« Reply #16 on: January 15, 2014, 07:10:48 pm »
Quote
The ball is in your court...
I won't argue anymore, but I'm gonna fix some mistakes.

Quote
Well, I think you are being  bit optimistic. We don't even have enough developers to get a Wireless NIC working (although that seems to be on the short list).
Four developers work on OS/4 kernel for years. Although removing the limit may cost some, because it is a lot of work, it can be accomplished.

Quote
Btw im writing this on a computer with 512 MB RAM
Me too. Although, that's how much memory in my laptop is installed right now, I'm gonna upgrade to 2 GiB (the maximum).

Quote
Very recently, the QSINIT project has demonstrated that using memory above the 32 bit addressing limit is possible. That is currently restricted to using it as a RAMDISK, but it has been demonstrated that it does work. More serious work is required to be able to use that memory more productively, and it is likely that programs would need to be modified to use it. Note that this is NOT the ideal solution, but, at the moment, it is the ONLY game in town.
It was obvious that using memory above the 32 bit addressing limit is possible, because you can use more than 4 GiB of memory on Windows 32-bit using ramdrive with PAE support by placing the swapfile on it.

Then, the first thing to do was to write a PAE supporting ramdrive, the hd4disk.add driver (which is acutally a standalone project, and it can be used without the Tetris bootloader).

Then, finally, to overcome the 4 GiB limit in the memory manager.

That is a very small cost comparing to rewriting the whole system on 64-bit, or comparing to implementing PAE directly in the memmanager, which would affect the drivers.

Quote
More limiting is the 500 meg memory space for each program (which includes shared memory space).
What limit?

Anyway, the memory management subsystem should be rewritten, and removing the 4 GiB barrier should be one of this task's subtargets.

dbanet

  • Guest
Re: Closed source parts of eComStation
« Reply #17 on: January 15, 2014, 07:18:39 pm »
I've tried to be as polite, as I could, and, I hope, the argument is settled.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4713
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Closed source parts of eComStation
« Reply #18 on: January 15, 2014, 09:23:56 pm »
There are a lot of things the eCS-OS2 is missing and it is hard to get a priority list since this priority change according the subject's needs.

One thread goes by there are people that think that eCS-OS2 is fine and we only need more drivers and keep patching it. Do not fix what it is not broken.

The other thread (me included) is try to clone as much of OS/2 as open source, even the components that are working. What it is working today will stop working tomorrow, and if we do not have the source code it is going to be harder to fix.

But,  no matter on which thread are you, the questions that we should ask are:
- Can you pull it off?
- Can you code the driver or the close source replacement?
- Can you hire someone that can do the development?
- Do you need help? What do you need?
- Can we pull it off to make a community raise fund for an specific driver or component?

The statement should be "I want to help" and "I can help with this or that".

So, who wants to do it? what do you require? I want to help any open source effort on this platform.
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: Closed source parts of eComStation
« Reply #19 on: January 15, 2014, 10:37:30 pm »
Quote
I won't argue anymore, but I'm gonna fix some mistakes.

That was not a challenge to you, specifically (and I am not arguing). It was challenge to the whole OS/2 community, to start helping out on some of these basic projects. It doesn't matter if you fix the 4GB problem (which is a non issue at this time anyway, even in 32 bit windows - it is only the server versions that use PAE, unless Win8 has started using it). It will take years for anybody to port enough software to use it, and new development seems to be a thing of the past.

More important, are things like support for the majority of NICs that exist today, and in the future. Without communication (especially wireless), nobody will be able to continue using eCS. It just won't do the job. It can't always do the job today, without wireless communications. There are keyboard problems, there are video problems, there are printer problems. Somebody had better start fixing all of them, or eCS will be as dead as IBM, and Microsoft, wishes it was.

Quote
Four developers work on OS/4 kernel for years.

Yes, and they are 4 developers who are wasting their time, if the more basic problems are not fixed soon. I don't want to discourage you, but these other, more basic, problems are going to kill the whole thing, before you get finished with what you are doing. Trying to get users back, after they go through the pain of going somewhere else, is not an easy job, as the current eCS project demonstrates. It is possible that we have already lost the critical mass to be able to maintain eCS as a viable project, and fixing the 4 GB barrier, today, would not prevent users from leaving, simply to get their WiFi adapter to work. Other things are going to become more important, in the next year, or two (USB 3.0, Bluetooth, WLAN), which haven't even been addressed, yet, although USB 3.0 is planned. You could have 64 GB of memory, that is usable, but that is not as useful as WiFi support, for the average power user, who would rarely, if ever, actually use 3 GB of memory, never mind 64 GB of memory. Personally, I find that I never use 2 GB of memory, unless I start a VBox, with a large memory assigned. That means that having a RAMDISK for the swap file, that will never be used, is not really going to help much.

What I am trying to get at, is that your 4 programmers could accomplish a lot more by working on other projects, that the average user will find useful, than to spend time messing around with breaking the 4 GB barrier, which won't do anybody any good, until a lot of other things get fixed (compilers, linkers). Those things are badly broken today, which limits porting, and new programming. anyway, and that discourages users from remaining in the OS/2 user group.

Quote
So, who wants to do it? what do you require? I want to help any open source effort on this platform.

It doesn't matter if these things are open source, or not. If they don't happen, eCS becomes less, and less, usable, and that means that more, and more, users must go elsewhere so they can do what needs to be done. Why would they even consider coming back? Sure, open source is desirable, but those who do the job need to make a living. We don't have corporate support, like Linux does, and most development is being done on a volunteer basis (even Linux is not 100% open source). Mensys has stepped up, and they have paid to get some necessary things done, but they don't have unlimited funds either. Mensys has committed to open sourcing what they pay for, and AFAIK, they have done that. What they haven't done, is make binaries available to those who don't pay for eCS. To me, that is a really good compromise. They do need to have a reason for users to buy eCS, and pay for Software Subscription, which pays for the development, as well as the wages of a couple of people who are doing a great job of holding it all together. I am pretty sure, that eCS is a non profit project, for Mensys, and they spend as much as possible on development, without running the company into bankruptcy. Is it enough? Probably not, and that is where the users need to step up, and do more to help out, if they want to keep eCS going.

The main problem is that there is not enough money, or people, to do any more than what is being done today. That is not enough to even keep up with the important stuff, never mind make advances that are not necessary, even though they may become necessary, eventually. Like it, or not, Mensys is the focal point, and if they find that they cannot cover their costs, eCS will be gone. That would leave OS/2, in all of it's forms, with no option but to pass into history.

dbanet

  • Guest
Re: Closed source parts of eComStation
« Reply #20 on: January 15, 2014, 11:07:49 pm »
Yes, and they are 4 developers who are wasting their time, if the more basic problems are not fixed soon.
Ok, that's enough for me. Didn't read more.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: Closed source parts of eComStation
« Reply #21 on: January 15, 2014, 11:38:55 pm »
As far as I know, the 4 GB barrier is built into X86. We use 32 bit pointers which are limited to 4Gbs and while most CPU's are now capable of using 64 bit pointers the CPU has to be put into a different state where 16 bit code no longer works and OS/2 still has too much 16 bit code to ignore, especially as much of the 16 bit code is device drivers.
All 64 bit capable CPUs also do PAE and as has recently been shown, os2ldr can be rewritten to access that extra 60GBs but it is kinda like bank switching (actually if I understand it, it's back to segments). You can access different 4 GBs banks or segments, great for a ramdisk but not really extending the address space visible to a program. Perhaps it is possible to give different programs their own address space so eg Firefox has access to 4GBs of real ram minus whatever the kernel needs along with what the hardware needs and this would be an improvement. How hard this would be I don't know. I do know that MS gave up on the idea for client versions of Windows as too many device drivers were not happy with it.
As far as I know the swap file is also limited by 32 bit types, probably signed so limited to 2 GBs which means putting the swap file on the ram disk can only help so far.
If someone is really dealing with large bitmaps, video files etc, they really have to move to a 64 bit operating system and purchase more memory

dbanet

  • Guest
Re: Closed source parts of eComStation
« Reply #22 on: January 16, 2014, 01:17:53 am »
As far as I know, the 4 GB barrier is built into X86. We use 32 bit pointers which are limited to 4Gbs and while most CPU's are now capable of using 64 bit pointers the CPU has to be put into a different state where 16 bit code no longer works and OS/2 still has too much 16 bit code to ignore, especially as much of the 16 bit code is device drivers.
All 64 bit capable CPUs also do PAE and as has recently been shown, os2ldr can be rewritten to access that extra 60GBs but it is kinda like bank switching (actually if I understand it, it's back to segments). You can access different 4 GBs banks or segments, great for a ramdisk but not really extending the address space visible to a program. Perhaps it is possible to give different programs their own address space so eg Firefox has access to 4GBs of real ram minus whatever the kernel needs along with what the hardware needs and this would be an improvement. How hard this would be I don't know. I do know that MS gave up on the idea for client versions of Windows as too many device drivers were not happy with it.
As far as I know the swap file is also limited by 32 bit types, probably signed so limited to 2 GBs which means putting the swap file on the ram disk can only help so far.
If someone is really dealing with large bitmaps, video files etc, they really have to move to a 64 bit operating system and purchase more memory

The whole address space, on OS/2, that you can use with A LOT of processes (programs), is limited to 4 GiB.
So, even if you have the swapping feature enabled and large enough hard disk, you cannot run 500 programs to allocate 500 GiB (one GiB per process).

While on Windows 32-bit, one program still cannot allocate more than 4 GiB, but you can swap on your hard disk and allocate 500 GiB with a lot of processes.

So, the trick is to put the swapfile not on the harddisk, but on the ramdrive, which would be able to map the memory above the 4 GiB limit using PAE without affecting the rest of the system.

So, that is the limit I am talking about. It could be overcomed by reimplementing the memory management subsystem in the OS/2 kernel.



By the way, the hd4disk.add driver, which actually jumps into PAE and provides the ramdrive (not the loader or, especially, QSINIT), works pretty fast (with the words of the developer) -- you can test it by yourself, or ask me to measure the r/w speed and report the results.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4713
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Closed source parts of eComStation
« Reply #23 on: January 16, 2014, 03:08:02 am »

More important, are things like support for the majority of NICs that exist today, and in the future. Without communication (especially wireless), nobody will be able to continue using eCS. It just won't do the job. It can't always do the job today, without wireless communications. There are keyboard problems, there are video problems, there are printer problems. Somebody had better start fixing all of them, or eCS will be as dead as IBM, and Microsoft, wishes it was.

Doug, let's do something about it.

Do you know any NICs developer we can hire, or can set a quote to create a Wireless open source driver? on which drivers should we start? Can you talk to him and try to set a fund, bounty, whatever to help out?
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: Closed source parts of eComStation
« Reply #24 on: January 16, 2014, 06:28:05 am »
Quote
Doug, let's do something about it.

Do you know any NICs developer we can hire, or can set a quote to create a Wireless open source driver? on which drivers should we start? Can you talk to him and try to set a fund, bounty, whatever to help out?

I would love to have the time to mess with that. Do you know any place where I can get some 48 hour days?   :(

The only qualified programmers, that I know about, are all working overtime, trying to get eCS 2.2 out the door, or get Firefox updated. There are probably many others who could do the job, if they were interested.

I have heard that it is not all that difficult to find an appropriate Linux driver, and plug it into the template that Mensys has provided. I started to look into doing it (wired NICs) myself, more than once, but never had the time to follow up (I do work on other projects, I do have a life, and I already don't get enough sleep without taking on more projects).

David A. is apparently scheduled to start on Intel wireless NICs, once eCS 2.2 is finished. He will probably follow that with Realtek wireless NICs (although he may not be interested after his bad experiences with Realtek wired NICs). After that, it is likely to be up to the users to provide drivers for the rest of the hardware that is out there. Somebody, with a little knowledge of software, and hardware, could make a nice hobby of making the drivers, once they figure out how to do it (apparently documented, in some detail). Of course, we can't expect anyone to run out and buy all of the possible devices (which is likely not possible anyway), so others need to provide testing, and semi-intelligent feedback, when something works, or doesn't work.

Michaelhz

  • Guest
Re: Closed source parts of eComStation
« Reply #25 on: January 16, 2014, 07:31:34 am »
After that, it is likely to be up to the users to provide drivers for the rest of the hardware that is out there. Somebody, with a little knowledge of software, and hardware, could make a nice hobby of making the drivers, once they figure out how to do it (apparently documented, in some detail).
Most probably these volunteers need access to the DDK, which is not available online. So it is hard for "hobby-programmers" to provide code for device drivers or even experiment with the existing code.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4713
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Closed source parts of eComStation
« Reply #26 on: January 16, 2014, 01:28:09 pm »
Most probably these volunteers need access to the DDK, which is not available online. So it is hard for "hobby-programmers" to provide code for device drivers or even experiment with the existing code.

You are right, but it is also good to remember that anybody that has eCS has the Developer Kit on the CD#2.

About the "IBM Device Driver Development Kit 2004" (different kit), I got permission from IBM to repost all the documentation on the EDM/2 (sorry but I still do not finish that migration). The sad thing is the license that the Device Driver Developer Kit has. It has a license that does not allow open source derivative work. But there is also a good thing, we have a lot of open source drivers that can be used as a templates.

It will be a good thing if someone can pin point me to the programs that are required to create drivers that came from the "Device Driver Developer kit" CD and have no replacement.
Do we still require the old Microsoft C to compile drivers? any thoughts?
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Michaelhz

  • Guest
Re: Closed source parts of eComStation
« Reply #27 on: January 16, 2014, 01:49:15 pm »
Do we still require the old Microsoft C to compile drivers? any thoughts?
The recent device drivers require openwatcom compiler (e.q. os2ahci), but of course are still depending on DDK and Toolkit.

Pete

  • Hero Member
  • *****
  • Posts: 1281
  • Karma: +9/-0
    • View Profile
Re: Closed source parts of eComStation
« Reply #28 on: January 16, 2014, 04:16:33 pm »
Hi All

Just to emphasize Dougs point about connectivity problems.

I recently purchased a 2nd user Dell E5500 laptop.

I installed eCS 2.2 beta 2 on it.

Bad move as I then found that there was no chance of getting the touchpad to work - who wants to carry an extra mouse around with them? - and even less chance of getting the wired or wireless nic work.

The wired nic is a broadcom netxtreme gigabit device that is too new to work with ye ancient b57.os2 nic driver and I could not, despite a lot of help from Doug and others, get genmac to work with the Windows 2K/XP drivers available.

The comments about trying genmac apply to the wireless nic, a Dell 1510 Wireless-N (made by Broadcom), as well as the wired nic.

I did email Broadcom and politely ask if there was any chance of them releasing the source code for the b57.os2 driver as it would be a good starting point in creating a native os/2 driver for current broadcom wired nics. Their response was a simple reply: "No support os/2". Thank You, Broadcom - I'll add you to my "hardware to be avoided" list.

So, I have a nice laptop with an eCS 2.2 install, no touchpad and no connectivity.

Solution options:-

1] Keep laptop and forget using eCS on it; Not an option as I bought a laptop so that I can carry on working if my wife wants to go to bed - our home office takes up a little bedroom space....

2] Sell laptop for close to what I paid for it and find an alternative laptop that *may* work.

3] However, I do not plan on spending lots of time buying and selling laptops in order to find something that works - and I'd rather have something reasonably current rather than a 10 year old laptop.

As you can guess from the fact that I asked broadcom for the b57.os2 source code I could have been interested in developing a wired nic driver.

What I find totally off putting is the amount of time I would need to spend getting to grips with the tools required for such development. It seems that I would not only have to relearn "c" - not a huge task in itself as my usual programming language is pascal which is closely related to "c" - but I would also have to discover what compiler and toolset is required and learn how to use them.

Most off putting.

Why do drivers need to be written in "c"?

What compiler is required?

What toolset is required?

How long will it take to get to grips with this development system bearing in mind that I'd probably only have, at most, 2 hours a day for 3 or 4 days in the week? - I suspect that by the time I got a working driver built the hardware would be obsolete...


I really could not see any obvious answers to the questions in 3] so have opted for 2]

If I was actually earning by building a nic driver I would be able to spend a little more time on the project but would still only manage between 16 to 20 hours a week - I just have too many other things going on in my life  :-)


Regards

Pete







Eugene Gorbunoff

  • Full Member
  • ***
  • Posts: 131
  • Karma: +1/-0
    • View Profile
Re: Closed source parts of eComStation
« Reply #29 on: January 16, 2014, 04:57:45 pm »
To Pete
* touchpad - did you tried disable and enable it again using Fn keys?
* WiFi - the users of Thinkpad apply patched BIOS to remove White list and insert supported Intel 4965AGN adapter.