Subversion Repositories wimsdev

Rev

Rev 6800 | Rev 6803 | Go to most recent revision | Show entire file | Ignore whitespace | Details | Blame | Last modification | View Log | RSS feed

Rev 6800 Rev 6802
Line 1... Line 1...
1 
WIMS' search engine and als
 1 
WIMS' search engine and als
2 
===========================
 2 
===========================
3 
 
 3 
 
4 
WIMS' search engine works in two stages:
 4 
WIMS' search engine works in two stages:
5 
 
 5 
 
- 
 
 6 
1) update of index files when server data is changed (module added...),
- 
 
 7 
   typically once a day.
- 
 
 8 
2) use of index files at each user's request to find some activities
- 
 
 9 
 
- 
 
 10 
 
- 
 
 11 
Here are some details :
- 
 
 12 
 
- 
 
 13 
1) update of index files
- 
 
 14 
===========================
6 
1) A series of scripts create a set of auxiliary files (generally
 15 
A series of scripts creates a set of auxiliary files (generally
7 
stored in ~/public_html/bases/sys/, see description further down) and
 16 
stored in ~/public_html/bases/sys/, see description further down) and
8 
a list of "keywords" (stored in ~/public_html/bases/site/).
 17 
a list of "keywords" (stored in ~/public_html/bases/site/).
9 
 
 18 
 
10 
(the scripts must be run in the order given here, as some files
 19 
(the scripts must be run in the order given here, as some files
11 
created on earlier stages are used in subsequent stages). In general
 20 
created on earlier stages are used in subsequent stages). In general
12 
the whole process is run by the script ~/bin/mkindex.
 21 
the whole process is run by the script ~/bin/mkindex.
13 
 
 22 
 
- 
 
 23 
* Firstly a series of 3 perl scripts (mkdomain,mkwgrp,modindclass),
- 
 
 24 
that ~/bin/mkindex.sh calls via ~/public_html/bases/sys/mkindex.sh :
- 
 
 25 
 
- 
 
 26 
- the programm ~/public_html/bases/sys/mkdomain.pl creates the lists
- 
 
 27 
  of domains from the graph in domain/domain with its translations
- 
 
 28 
  (domain/domain.$lang) and in json format (english) to be used for
- 
 
 29 
  completion in modtool properties
- 
 
 30
14 
- the perl program ~/public_html/bases/sys/mkwgrp.pl reads the INDEX
 31 
- the perl program ~/public_html/bases/sys/mkwgrp.pl reads the INDEX
15 
  files of all the modules on the site and generates
32 
  files of all the modules on the site and generates
16 
 
 33 
 
17 
  - keywords (in format .json) to be used for completion in the search
 34 
  - keywords (in format .json) to be used for completion in the search
18 
    engine)
 35 
    engine)
Line 26... Line 43...
26 
  not used for the moment. The keywords in these "keywords file" are
 43 
  not used for the moment. The keywords in these "keywords file" are
27 
  exactly those in the variable keywords (or keywords_$lang if it
 44 
  exactly those in the variable keywords (or keywords_$lang if it
28 
  exists), doing it with the following rules: taking keywords_$lang if
 45 
  exists), doing it with the following rules: taking keywords_$lang if
29 
  it exists, or keywords (whatever it is a $lang-module or not).
 46 
  it exists, or keywords (whatever it is a $lang-module or not).
30 
 
 47 
 
31 
- the programm ~/public_html/bases/sys/mkdomain.pl creates the lists
 - 
 
32 
  of domains from the graph in domain/domain with its translations
 - 
 
33 
  (domain/domain.$lang) and in json format (english) to be used for
 - 
 
34 
  completion in modtool properties
 - 
 
35
- 
 
36 
- the program ~/public_html/bases/sys/modindclass.pl creates the lists
 48 
- the program ~/public_html/bases/sys/modindclass.pl creates the lists
37 
  of keywords coming from the example classes in
 49 
  of keywords coming from the example classes in
38 
  ~/public_html/bases/class as well as the files author,
 50 
  ~/public_html/bases/class as well as the files author,
39 
  description, language, level, title (no ranking is done).
 51 
  description, language, level, title (no ranking is done).
40
52
41 
- the program "modind" reads
53 
* Secondly the binary program "modind" (compiled from ~/src/Misc/modind.c) reads
42 
 
 54 
 
43 
  -- the INDEX files of all the modules on the site
55 
  -- the INDEX files of all the modules on the site
44 
  -- the auxiliary files in ~/public_html/bases/sys/ (see description
 56 
  -- the auxiliary files in ~/public_html/bases/sys/ (see description
45 
     below)
 57 
     below)
46 
 
 58 
 
Line 60... Line 72...
60 
  ~/public_html/bases/sys/, e.g. wgrp
 72 
  ~/public_html/bases/sys/, e.g. wgrp
61 
 
 73 
 
62 
  -- separately "modind" reads also the files in
 74 
  -- separately "modind" reads also the files in
63 
  ~/public_html/bases/sys/sheet and do the same type of works
 75 
  ~/public_html/bases/sys/sheet and do the same type of works
64 
 
 76 
 
- 
 
 77 
 
- 
 
 78 
2) use of index files
- 
 
 79 
===========================
65 
2) The script ~/public_html/modules/home/search.proc (called by the
 80 
The script ~/public_html/modules/home/search.proc (called by the
66 
"Search" form) reads the lists above, do the actual search in such
 81 
"Search" form) reads the lists above, do the actual search in such
67 
lists and displays the modules found. It reads also the files of
 82 
lists and displays the modules found. It reads also the files of
68 
~/public_html/bases/sys/class and ~/public_html/bases/sys/sheets
 83 
~/public_html/bases/sys/class and ~/public_html/bases/sys/sheets
- 
 
 84 
 
- 
 
 85 
 
- 
 
 86 
 
- 
 
 87 
More technical details about both stages
- 
 
 88 
========================================
69 
 
 89 
 
70 
In both stages files in this directory ~/public_html/bases/sys/ (see comments
 90 
In both stages files in this directory ~/public_html/bases/sys/ (see comments
71 
below)(suffix.$lang for example, but see upper remark) are used to
 91 
below)(suffix.$lang for example, but see upper remark) are used to
72 
process the keywords present in the modules' INDEX files.  Each
 92 
process the keywords present in the modules' INDEX files.  Each
73 
"search language" has its own series of files.
 93 
"search language" has its own series of files.