OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: Paul Smedley on September 18, 2020, 04:55:35 am

Title: gImageReader - qt5 OCR application
Post by: Paul Smedley on September 18, 2020, 04:55:35 am
Hi all,

Per some discussion on the eComStation yahoo group mailing list, I've built gImageReader - a Qt5 application that uses the tesseract OCR engine.

This seems to work - albeit the GUI is a little painful to use - at least in my VM.

I've tried to document the dependencies below - no promises I didn't miss anything :)

gImageReader-qt5.exe + qtspell0.dll + quazip51.dll - http://smedley.id.au/tmp/gimagereader-3.3.1-os2-20200918.zip

Requirements:
libcx0.dll -
poppq51.dll - (yum install poppler-qt5)
qt5 DLL's - (yum install qt5)
usb100.dll - (yum install libusb1)

A copy of the relevant language file for tesseract (the OCR Engine) - see https://tesseract-ocr.github.io/tessdoc/Data-Files.html for downloads.

Tesseract language files can go in /tesseract/share/tessdata - or edit preferences in gImageReader to specify a local directory.

Cheers,

Paul
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on September 18, 2020, 06:40:43 pm
 Hello
 I have tried very little. But it seems to me to work fine.
Since I didn't have a scanner handy, I created the text in open office,
 and copied it with pmview to png. And I attach the result. I only failed the character O º
 I'm also not sure if I have selected the correct language version. :)
Saludos
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on September 19, 2020, 01:01:16 am
Slight update @ http://smedley.id.au/tmp/gimagereader-3.3.1-os2-20200919.zip

Now looks in (application dir)/share/tessdata for language files which is a bit more OS/2 friendly
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on September 19, 2020, 09:19:55 am
[D:\gimagereader-3.3.1-os2-20200919]GIMAGEREADER-QT5
SYS1804: The system cannot find the file QTSPELL0.

Saludos
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on September 19, 2020, 09:40:52 am
[D:\gimagereader-3.3.1-os2-20200919]GIMAGEREADER-QT5
SYS1804: The system cannot find the file QTSPELL0.

Those were in the original zip, I've now added them to the 0919 zip
Title: Re: gImageReader - qt5 OCR application
Post by: Neil Waldhauer on October 04, 2020, 07:44:42 am
I can load the application, but I can't seem to get it to scan.

I uploaded a screen shot
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on October 04, 2020, 08:24:09 am
I think you're missing the 'Recognize all English' button in the toolbar.....

Title: Re: gImageReader - qt5 OCR application
Post by: Neil Waldhauer on October 04, 2020, 05:10:02 pm
Yes, that was it. I have to move this to a faster machine. But it is kind of recognizing text and displaying it in the output window. I don't have much experience with OCR programs.

If I find a way to recognize a source with multiple columns, I'll edit this post.
Title: Re: gImageReader - qt5 OCR application
Post by: Neil Waldhauer on October 16, 2020, 10:34:36 pm
The speed problem seems to be in QT5, and a fix is expected. I have carved up some of my multi-column files into single-column files, and they seem to OCR fairly well, but a few misses that seem to be due to the missing spelling files.

How would I add a dictionary?
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on October 16, 2020, 11:40:17 pm
The speed problem seems to be in QT5, and a fix is expected. I have carved up some of my multi-column files into single-column files, and they seem to OCR fairly well, but a few misses that seem to be due to the missing spelling files.

How would I add a dictionary?

Until the downloading is fixed - copy the appropriate language file from https://github.com/wooorm/dictionaries/tree/main/dictionaries into ./share/hunspell

For others, the qt5core fix is linked from https://github.com/bitwiseworks/qtbase-os2/issues/121

Cheers,

Paul
Title: Re: gImageReader - qt5 OCR application
Post by: Neil Waldhauer on October 18, 2020, 04:38:05 am
The speed problem seems to be in QT5, and a fix is expected. I have carved up some of my multi-column files into single-column files, and they seem to OCR fairly well, but a few misses that seem to be due to the missing spelling files.

How would I add a dictionary?

Until the downloading is fixed - copy the appropriate language file from https://github.com/wooorm/dictionaries/tree/main/dictionaries into ./share/hunspell

For others, the qt5core fix is linked from https://github.com/bitwiseworks/qtbase-os2/issues/121

Cheers,

Paul

I copied all the files into ./share/hunspell, but it still doesn't know English. I wonder if there should be a sub-directory structure to support different languages?
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on October 18, 2020, 04:30:50 pm
Neil,
I attach a photo so you can see where the language is. Look at the upper right corner,
 below the buttons in the window.

 Paul,
I attach another photo with a translation of this website, in about 1 second. With this qt5core.dll
Directory of C:\USR\LIB

18/10/19  6:37      4.335.130    342 a---  Qt5Core.dll
I don't know if this is a long or a short time, but this works perfect. I have only detected two failures.
1- With the external usb mouse, in usb3.0 the mouse crashes, and I have to disconnect and reconnect it.
Or connect it to a usb 2.0 port
2-When I closed the program it showed me this error in the command prompt,
QTextCursor::setPosition: Position '811' out of range
The keyboard got stuck, and when closing the command prompt, the keyboard returned to normal.
Saludos
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on October 18, 2020, 04:51:46 pm
For others, the qt5core fix is linked from https://github.com/bitwiseworks/qtbase-os2/issues/121

That may be an option, but I think it is better to try to learn to understand how memory and swapper work.
I don't know yet, but doing this works very well for me.
You can try this, rename the two lines of the config.sys
1-  rem MEMMAN=SWAP,PROTECT
and
2-  rem SWAPPATH=C:\OS2\SYSTEM 2048 2048

Copy this lines after that:

MEMMAN=SWAP,PROTECT,COMMIT
SWAPPATH=C:\OS2\SYSTEM 402048 402048
SWAPPATH=C:\OS2\SYSTEM 311286 311286
MEMMAN=SWAP,PROTECT
SWAPPATH=C:\OS2\SYSTEM 402048 402048
SWAPPATH=C:\OS2\SYSTEM 311286 311286
MEMMAN=SWAP,PROTECT,COMMIT
SWAPPATH=C:\OS2\SYSTEM 402048 402048
SWAPPATH=C:\OS2\SYSTEM 311286 311286
MEMMAN=SWAP,PROTECT
SWAPPATH=C:\OS2\SYSTEM 402048 402048
SWAPPATH=C:\OS2\SYSTEM 311286 311286

Doing this increases my memory by about 200mb.
Try it, the speed, and lightness is never seen before.
I do not need any patch for the processors for now.

Set the virtualaddresslimit to a value that works for you, and you're done.
I now have 1688, but I still have this under observation.
Saludos

Title: Re: gImageReader - qt5 OCR application
Post by: Andreas Schnellbacher on October 18, 2020, 09:53:24 pm
The last statement wins. There's no need for the others.

The only change you've made is to increase both the maximum and the initial value of SWAPPER.DAT by 300 MB. I don't see the benefit by that.

About VAL: I have mine set to 2560. The physically recognized memory is just 2 GB. That enables me to load e.g. the Mozilla DLLs high and increases the time before the lower shared memory exhausts.

To check the four memory arenas, use either Alex' "mem /v" or Andrey's SystemLoad (sl.exe), module "DosQuerySysInfo".
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on October 19, 2020, 05:24:28 pm
The last statement wins. There's no need for the others.
Thanks,
Quote
The only change you've made is to increase both the maximum and the initial value of SWAPPER.DAT by 300 MB. I don't see the benefit by that.
I have not looked at it as a number of megabytes, I have looked at it as a number of compatibility with all programs.
All comments are in this tickect, start reading at the end.
https://mantis.arcanoae.com/view.php?id=1800
Quote
About VAL: I have mine set to 2560.
I like it, but I keep checking it.
Saludos
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on October 20, 2020, 12:36:32 am
For others, the qt5core fix is linked from https://github.com/bitwiseworks/qtbase-os2/issues/121

That may be an option, but I think it is better to try to learn to understand how memory and swapper work.


The qt5 cpu usage has nothing to do with memory and swapper - it's a bug.

Cheers,

Paul
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on October 20, 2020, 10:42:05 pm
Yes, you are correct.
Do you have any way to quickly repeat the error of the processors?
Saludos
Title: Re: gImageReader - qt5 OCR application
Post by: roberto on October 21, 2020, 02:30:42 pm
Shortly I think I'll make an update on this,
 you did not believe it.
As I advance some minimum memory levels, and another increase in free use memory.
You will see the super warp speed.
I still have to check the processor failure, but I think it works without problems.

[C:\]MEM /V

Total physical memory:      3 959 MB
Accessible to system:       3 063 MB
Additional (PAE) memory:      896 MB

Resident memory:              170 MB
Available virtual memory:     457 MB                 ***runing nice ****

Available process memory:
  Private low memory:         100 MB
  Private high memory:      1 792 MB
  Shared low memory:            0 MB
  Shared high memory:         421 MB

[C:\]

saludos
Title: Re: gImageReader - qt5 OCR application
Post by: Eduardo Vila Echagüe on October 27, 2020, 03:17:05 pm
Paul,

You have left unanswered Neil's comment about dictionaries. In the URL you mention, all the languages use files with the same names. You cannot copy them together to the hunspell folder. And then, how are them installed? The readme says using npm, but npm is not available in OS2.

How would I add a dictionary?
[/quote]

And what are the dictionaries for? I have read documents in English, French and Spanish quite well, with no dictionaries installed.

Eduardo

Until the downloading is fixed - copy the appropriate language file from https://github.com/wooorm/dictionaries/tree/main/dictionaries into ./share/hunspell

For others, the qt5core fix is linked from https://github.com/bitwiseworks/qtbase-os2/issues/121

Cheers,

Paul
[/quote]

I copied all the files into ./share/hunspell, but it still doesn't know English. I wonder if there should be a sub-directory structure to support different languages?
[/quote]
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on October 29, 2020, 09:26:38 am
Hi Eduardo,

You have left unanswered Neil's comment about dictionaries. In the URL you mention, all the languages use files with the same names. You cannot copy them together to the hunspell folder. And then, how are them installed? The readme says using npm, but npm is not available in OS2.

How would I add a dictionary?
And what are the dictionaries for? I have read documents in English, French and Spanish quite well, with no dictionaries installed.


If I haven't answered, it's because I don't know. I compiled this software after a discussion on OCR in a mailing list. I have no real need for it myself. In terms of the dictionary, I assume after the OCR, if a dictionary is installed,  it will spell check to improve the accuracy.

Cheers,

Paul
Title: Re: gImageReader - qt5 OCR application
Post by: Dave Yeo on October 29, 2020, 03:34:29 pm
Is there a configure/cmake option to use the system Hunspell?
Title: Re: gImageReader - qt5 OCR application
Post by: Neil Waldhauer on October 29, 2020, 08:38:55 pm
Hi Eduardo,

You have left unanswered Neil's comment about dictionaries. In the URL you mention, all the languages use files with the same names. You cannot copy them together to the hunspell folder. And then, how are them installed? The readme says using npm, but npm is not available in OS2.

How would I add a dictionary?
And what are the dictionaries for? I have read documents in English, French and Spanish quite well, with no dictionaries installed.


If I haven't answered, it's because I don't know. I compiled this software after a discussion on OCR in a mailing list. I have no real need for it myself. In terms of the dictionary, I assume after the OCR, if a dictionary is installed,  it will spell check to improve the accuracy.

Cheers,

Paul

I asked for just that reason. Recognition is pretty good, but if spell check were added, it might have been better. I'm not really in need of OCR, but if I was, this program is very promising. I tried some of the same inputs for Recognita and I did not get good results. It is hard to prepare files for input to Recognita. gImageReader takes almost anything.
Title: Re: gImageReader - qt5 OCR application
Post by: Paul Smedley on October 30, 2020, 06:05:59 am
Hi Eduardo,

You have left unanswered Neil's comment about dictionaries. In the URL you mention, all the languages use files with the same names. You cannot copy them together to the hunspell folder. And then, how are them installed? The readme says using npm, but npm is not available in OS2.

How would I add a dictionary?
And what are the dictionaries for? I have read documents in English, French and Spanish quite well, with no dictionaries installed.


If I haven't answered, it's because I don't know. I compiled this software after a discussion on OCR in a mailing list. I have no real need for it myself. In terms of the dictionary, I assume after the OCR, if a dictionary is installed,  it will spell check to improve the accuracy.

Cheers,

Paul

I asked for just that reason. Recognition is pretty good, but if spell check were added, it might have been better. I'm not really in need of OCR, but if I was, this program is very promising. I tried some of the same inputs for Recognita and I did not get good results. It is hard to prepare files for input to Recognita. gImageReader takes almost anything.

I'll try and look at this again at some point, but right now, my priority is fixing my updated Uniaud32 build...