• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

Tesseract Font Box Editor (PM/GUI)

Started by jep, 2009.01.18, 12:15:56

Previous topic - Next topic

jep

#15
Quote from: warpcafe on 2009.02.09, 11:18:53
Hi all,

----->8=======
=======8<-----

Goodness! If someone is able to do that, can he/she please-please have a look at the famous "Warp4 style notebook" topic and let us know how to patch /reassemble DrDialog such that it is able to use the "warp" (more contemporary) style for notebook controls? :)

Cheers,
Thomas

While it's easy to change all the built in images (and e.g. rearrange the debug window) inside the executables, it's more difficult to touch the code to set the right bit to make it turn on the Warp4 notebook style on in DrDialog. I tried as some discussion suggested and was able to get the Warp4 notebook look, but the tabs didn't work right (Very narrow, label not visible or floating around), nor did the tabs respond to input. :-<
Unfortunately, I guess that the notebook creation must reside far down, inside the code.

"Another look for DrRexx"

//Jan-Erik

jep

Hi,

I've now updated eTFBE some more, as a matter of fact it seem to be able to produce all files required, except the one to create a dictionary! You just have to move the files to the right place in a subfolder to tesseract and rename them!
(The files created haven't been tested, but seem to be ok and should therefore work)

Have enabled output to ".tr", and the various file formats AND added code to save the ambiguities to file.

You'll find most options under the OCR menu when you load a project except the one to save Ambiguities, it remain under File -> Save as -> Ambiguities.

"Files that eTFBE can create for you"

What remain to be done (with your help):
Allow one to select target language and use it in the project
Create image from text
Fix various isseus in the GUI
Handle output from some of the tesseract apps.
Add code to convert a text file with words to a dictionary
etc.

//Jan-Erik

DrDialog can create executables for you! (out of the DrDialog .res file)
Just drag & drop the .res file in the .zip-file to the program object REStoEXE in the DrDialog folder on you desktop and you're ready to go.

warpcafe

Hi jep,

Quote from: jep on 2009.02.19, 07:50:52
What remain to be done (with your help):
...
Create image from text
...

I haven't followed in detail... are you talking about creating a graphics image file (e.g. ".BMP") from a plain-text file (e.g. ".TXT") ? Or to put it that way: "Make a picture of a text page"? In that case -and with regards to DrDialog or rexx in general- I would recommend to use the rexxgd package. IIRC there are functions to create graphics (empty in your case) and insert text into them. All by calling rexx DLL procedures. ;)

Cheers,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

jep

#18
Hi warpcafe,

Quote from: warpcafe on 2009.02.19, 12:06:08
Hi jep,

I haven't followed in detail... are you talking about creating a graphics image file (e.g. ".BMP") from a plain-text file (e.g. ".TXT") ?

Yes, TXT (or UTF-8) -> BMP ( -> TIF, see below )
or use the editor to write the text down...
or take text from the clipboard...
and create the image from that

Quote from: warpcafe on 2009.02.19, 12:06:08
Or to put it that way: "Make a picture of a text page"?

Yes, exactly, why not use text we already have to create tons of training images!
Note: eTFBE can use the original text file for the training procedure as it depend on either human input or help from a text file.

Quote from: warpcafe on 2009.02.19, 12:06:08
In that case -and with regards to DrDialog or rexx in general- I would recommend to use the rexxgd package.

Yes, rexxgd by Mark Hessling...

Quote from: warpcafe on 2009.02.19, 12:06:08
IIRC there are functions to create graphics (empty in your case) and insert text into them. All by calling rexx DLL procedures. ;)

Cheers,
Thomas

Can you show us how, Thomas?!
Just take into account the "fontmetrics"¹ calculations for each row.

Anyone else may try as well! Get rexxgd!

You can then convert it to TIF with rxImgSze/GBM combo with
call rxImageSize img_bmp_extension, img_tif_extension
afterward

I'll look at the dictionary in the mean time... I probably have to find a way to convert all rows to UTF-8 before feeding it to tesseract.
//Jan-Erik

_____________
¹)  Length of the text for the specified font on the drawing surface.

Batchheizer

Hi Jep,
thanks for your work on the Font Box Editor. But I did not get it started (neither from DrDialog nor the created Exe). I do not know the reasons (and will not discuss this issue). I would like to test the program and help to improve it. PLEASE distribute it as a package (WPI or ZIP) that does not need any other special software and runs out of the box.

jep

Hi,

I'll package the files and post the zip tonight (CET) if not sooner.

//Jan-Erik

jep

Hi,

please see attached -zip-file. It contain the files you need, except Tesseract and GBM that you should have on you computer already anyway.

Note!
eTFBE doesn't run a second time unless you restart your computer due to that it can't load the image control and the main window doesn't appear. I don't have a clue why, but someone else may know what cause the problem and how to solve it. If you know how to solve it, please take the time to reply here and tell me/us what to do to fix it.

Info:
1) Use the menu option "Close" to close the image and the editor before you open another image you plan to use tesseract with.
2) All images you open will be moved to the project you create at the beginning. (See previous instruction).
3) Save your work once in a while, especially if you've edited text and plan to open another file.
4) eTFBE remember and load the last image and box file you worked with whenver you open a project.
5) The function to Cluster files, list and use all .tr-files it can find in the project folder, the same goes for .box-files to create the character set.

//Jan-Erik

Batchheizer

Hi Jep!
It runs! Loading takes a long time, but it seems to work. There is lot of harddisk-access on loading the program. Why? Does it search for something?

I will make some tests next week. Thanks for your work.

jep

Hi Batchheizer,

Yes, it's a strategic choice I've made so you wouldn't have to configure it by hand. It tries to find what it need on it's own and report to you what may be missing or doesn't seem to work.

That's what I consider user friendly, the first time you run it, it searches for all files needed such as tesseract on your computer and that may take a moment. It store that info into the Extended Attributes (EA) of eTFBE.cfg (it doesn't write into the file) and start the application. (See more info at the bottom)

The next time you run eTFBE, it first tries to read the EA from eTFBE.cfg.
If it can find the info and the check that file exist succeeds then it doesn't have to search further (as the first time) and can continue with the next procedure right away. That way it'll load and run faster though be able to handle changes in your system.

So, if you move tesseract or delete eTFBE.cfg it'll take longer to start again as it search for files. Also note that information about position and sizes of the windows are stored in eTFBE.cfg (The project folder instead if you load a project). Please see the file eTFBE.REX and the startup routine (RxInit) for more information how I've designed it.

//Jan-Erik

========
Supporting functions that load right, appear with version info in the About box
Rename a .dll and try how eTFBE behave.
If it can't find tesseract (or any other part) a message box should appear that tell you to get the the missing function.
The About box should then appear and show you where to get the missing function/dll
It should even show you the latest known location where to get it and if you click the link your browser will load the page so you can read more about it and download what's missing.

jep

Hello,

I've begun to write the code for word2dawg to convert a wordlist to a suitable format.
I've decided to add support for OpenOffice.org dictionaries since it support many languages and they're hopefully quite complete. It's quite easy to extract the info needed, clean the data (remove extra info from each word) and convert it to UTF8. The problem is that word2dawg doesn't run, not even with just a few words in the file, something that cause alot of problems. I've just downloaded the code and can see that it must be the initial loading (allocation, "emalloc") that doesn't handle it right.

Let's hope that Paul Smedley or someone else can come up with a solution to what might be the problem.

I'll add support for plain text and UTF8 files as well, but word2dawg has to be fixed before we can expect it to produce the right files.

//Jan-Erik

fbakan

Quote from: jep on 2009.02.26, 13:48:21

Let's hope that Paul Smedley or someone else can come up with a solution to what might be the problem.


I sent you a new compiled version of word2dawg attached in a mail.
This version worked here (I couldn't check the output, but at least it doesn't crash anymore and it produces a file)
Did you miss my mail?

best regards
Franz

jep

Hello Franz,

Quote from: fbakan on 2009.02.26, 19:52:26

I sent you a new compiled version of word2dawg attached in a mail.
This version worked here (I couldn't check the output, but at least it doesn't crash anymore and it produces a file)
Did you miss my mail?

best regards
Franz


Thank you for you help Franz, I really appreciate it.  :)
I'll lookout for it in my inbox whenever it may arrive!

//Jan-Erik
One step futher, one step closer to completing this topic

jep

Hi all,

I'm now making progress with the wordlist2dawg app that Franz wrote.

I need your help though to handle a couple of things, but you don't have to write any code, just come up with ideas.

Language Codes
I've got a file with the language codes that they want us to use for to represent each language, but what is the most efficient way to:
a) Load the text
b) Present the loooOOOoong list
c) Make it easy for the user to select the right code without scrolling toooOOOo much
Example: ISO-639-2 Codes to represent each langauge

The info below may require understanding for how rexx behave.

CodePages present within the spelling dictionaries
One ".aff" (inside the .oxt-zip-file) may contain a row like "SET IS0-8859-15", while the other contain "SET ISO-8859-2" and a third something completley different, but all has to be converted to UTF-8 to work with tesseract. It's "easy" to get that info from there, but then the problem begin.

What's the most efficient way to find the codepage to use for rxuls that each dictionary as each may contain different encodings?
I have a list with matching codepages, but how do one load it (a modified/adapted version to rexx) and get the right info/match from it with as little effort as possible?
Alex Taylors list of Codepages in OS/2-eCS

//Jan-Erik

warpcafe

Hi Jan-Erik,

Quote from: jep on 2009.03.06, 14:11:46
..., but what is the most efficient way to:
a) Load the text
b) Present the loooOOOoong list
c) Make it easy for the user to select the right code without scrolling toooOOOo much

Well it depends - you want to go for alpha-3 or alpha-2?
With alpha-2 the list will be considerably smaller. Another way would be to select in two steps like country first, then language but that would need a list of combinations allowed (e.g. Switzerland -> ch/de/it...) also a deductive preselection based upon the "lang" setting could ease the selection for the user.
A loooong list is not a problem if you can incremetally "type" in it to set the cursor. OS/2 hasn't got that I'm afraid. ;) It also depends if that is a one-time, on-demand or regular (for each file) process... perhaps use a "wizard"-thing that does the trick in two or more steps. That's better than a single spot which is cluttered with controls and listboxes.

Quote from: jep on 2009.03.06, 14:11:46
I have a list with matching codepages, but how do one load it (a modified/adapted version to rexx) and get the right info/match from it with as little effort as possible?

I'm context-heavy enough I guess. Can you give an example with files (or records)? Perhaps then I can give a suggestion based upon what functional approach is best/fast. We can alos think about transforming the information into something more machine-suited to find better/faster algorithms...
You know with rexx... the string manipulation functions are awesome (also when combined with stems) and one can do nifty tricks with that as long as we know what we would need to achieve. ;)

Cheers,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

jep

Hi,

Quote from: warpcafe on 2009.03.06, 17:29:10
========8<---------
--------->8========
Well it depends - you want to go for alpha-3 or alpha-2?
The tesseract documentation says one should use Alpha-3 so all will know what we mean then.
A code for Alpha-2 may also not be available for all languages.

Example of what Alpha-3 and Alpha-2 mean below:
Alpha-3Alpha-2Language name
abkabAbkhazian
ace Acehnese, Achinese
ach Acoli
ada Adangme
ady Adyghe; Adygei
aaraaAfar
etc.

Quote from: warpcafe on 2009.03.06, 17:29:10
With alpha-2 the list will be considerably smaller. Another way would be to select in two steps like country first, then language but that would need a list of combinations allowed (e.g. Switzerland -> ch/de/it...) also a deductive preselection based upon the "lang" setting could ease the selection for the user.

Yes, but even a 2 step one doesn't quite reduce the number of rows enough that has to be handled.

What do you mean with "a deductive preselection based upon the "lang" setting"?

Do you mean something like that it should extract the codepage set on the computer and assume that it may be the first choice?
Yes, I think that it's very likely to eliminate a lot of unnecessary loading procedures, very good thinking!

Hmm, one may even split it up into several files so that it only has to load the file that correspond to that info in the first run and load another if required.

Quote from: warpcafe on 2009.03.06, 17:29:10
A loooong list is not a problem if you can incremetally "type" in it to set the cursor. OS/2 hasn't got that I'm afraid. ;) It also depends if that is a one-time, on-demand or regular (for each file) process... perhaps use a "wizard"-thing that does the trick in two or more steps. That's better than a single spot which is cluttered with controls and listboxes.

Yes, seen a few compiled applications that try that approach in OS/2-eCS. Hmm, would be interesting if someone could come up with a smart and lightweight approach. It depend on how fast one can load, preload or fake the next part and present it to the user.

It would interesting if we could try something else before we give in and use a good and safe wizard-approach though.

Quote from: warpcafe on 2009.03.06, 17:29:10
========8<---------
--------->8========
I'm context-heavy enough I guess. Can you give an example with files (or records)? Perhaps then I can give a suggestion based upon what functional approach is best/fast. We can alos think about transforming the information into something more machine-suited to find better/faster algorithms...

It's easy to extract the text that define what encoding to use as mentioned by the SET statement mentioned earlier, from the dictionary .oxt-file, such as ISO-8859-1, Latin 1, Latin 2, Latin 3, Latin 4 etc.

The function that has to use that information in rxuls does however not like it in that form, so one has to convert it to something else that represent the codepage in plain numeric form. 639, 1004, ... 850 etc. That info is available and can be placed in a file that the app read in some way, but there are a lot of possible encoding-pairs to search through.

What I need to do is to take e.g the text "Latin 4" and find the matching needle "1257" (or the OS/2 equivalvence) in a haystack (Alex Taylors list of Codepages in OS/2-eCS), I guess. :P

Coding geek talk:
The SET-statement is present within the first 1 to 20 variables in a stem that one get with the unzip procedure


The time to spend on that is neglectable while searching in a text file for a match that return a value, take many times longer.

One can preload things, make it look fast (progressbar to obscure that it take time) or just fake things.
The thing is that it has to be selected only once for a project, so it shouldn't be a huge thing, but the user should be helped by it to select/set the correct three-letter code and not find it to be in the way.

Coding geek talk:
If one could have a variable for each key-value-pair that would return the right answer right away?
"Latin 4" could perhaps be transformed into the variable latin_4 that would contain the value 1257 (that b.t.w. is Polish) that could be used later on?
or
something so one doesn't have to preload or search through the huge list and many codepages


Quote from: warpcafe on 2009.03.06, 17:29:10
You know with rexx... the string manipulation functions are awesome (also when combined with stems) and one can do nifty tricks with that as long as we know what we would need to achieve. ;)

Cheers,
Thomas

Yes, that's what so awesome with rexx!

Thank you for your suggestions Thomas!



The attached example contain some code to extract words from an .oxt file, but assume that it's encoded with the codepage 1004. It may be different for your language and is mentioned within the first lines of the *.aff-file also present within the .oxt-file. The code assume that you have the file "ooo_swedish_dict_1.29.oxt" present in D: if you don't provide the path and file name as a parameter to the script. You also need rxuls form Alex Taylors page to test the code.

//Jan-Erik