Author: S. Hsiung.
iqats.zip Download everything here - binaries, code, documentation.
IQATS (Intelligent Question and Answer Test Summarizer) is natural language proccessing program that creates questions and answers given a piece of text (a sentence) through a simple pattern matching algorithm.
 ScreenshotsIQATS in action (1) (screen01.jpg)
Implementation and File Organization of IQATS
Example Input: Bill Clinton was the President in 1993 1. ((proper-noun ("Bill")) (proper-noun ("Clinton")) (verb-to-be ("was")) (determiner ("the")) (proper-noun ("President")) (preposition-proximity ("in")) (number ("1993")) (eos ("."))) - Grammar.dat, Grammar.lst. The first parsing is simple (function: conv-sent-fully), phrase markers such as verb-to-be, determiner, preposition-proximity are listed as categories in Grammar.dat. IQATS searches for the words that are in these categories, then parses them. Those words were chosen to indicate something that would later suggest a question to ask. For example "was" in this case is already semantically defined as something that is designating an object to also be another object. (Note: all data files are in plain-text and can be edited by the user). Phrase markers such as proper-noun, number require another form of identification on parsing. They are procedural based and can only be edited inside the source code. Proper-noun are recognized as words that are not in Grammar.dat but are capitalized. 2. ((noun-phrase (proper-noun ("Bill")) (proper-noun ("Clinton"))) (verb-to-be-phrase (verb-to-be ("was"))) (noun-phrase (determiner ("the")) (proper-noun ("President"))) (preposition-time-phrase (preposition- proximity ("in")) (number ("1993"))) (eos ("."))) - Identify.dat, Identify.lst, (function: group-all-sentence). In IQATS, a prepositional phrase can contain another prepositional phrase. However this may provide some inconvenience because the pattern matcher only includes 2nd level phrase markers. For example, in the 2nd leaf, we have (proper-noun ("Clinton")) inside a noun-phrase, the noun-phrase would be 2nd level, while 'proper-noun' would be 1st Level, (visualizing it as a tree would be helpful). Notice the distinction of a preposition-time-phrase with just another prepositional phrase. In the data files, phrase-markers from the last parsing as well as this parsing are used to identify with the present parsing. Therefore, order is certainly an issue. Identify.dat also distinguishes phrase leaves which are required, and ones that can be included, but are not necessarily indicators of that marker. This is always the second value under the category (refer to Indentify.dat in the appendix or exhibit). It also has a flexibility factor of 'unknown' words. 3. ((person (noun-phrase (proper-noun ("Bill")) (proper-noun ("Clinton")))) (verb-to-be-phrase (verb-to-be ("was"))) (object (noun-phrase (determiner ("the")) (proper-noun ("President")))) (time (preposition-time- phrase (preposition-proximity ("in")) (number ("1993")))) (eos ("."))). - Semantic.dat, Semantic.lst (function: apply-semantics-sentence). This parsing is semantic; most of these markers are used directly for pattern matching. For example (cat-listing-basic sentence) would return (person verb-to-be- phrase object time). Note also that there are two types of objects (but not in this case), object and object- minor. Object designates a noun-phrase and object-minor designates a recipient- a prepositional phrase or a direct object. 4. (person verb-to-be-phrase object time) (manip.dat) Question: (verb-to-be-phrase object time) (manip.ord) Answer: (person) (answer.dat). (("Who" "was" "the" "President" "in" "1993" "?") ("Bill" "Clinton")). (Note: Question words such as who are in apply.dat, IQATS also stores the need to change from past->present tense inside question.dat). All the patterns (above) are stored in data files to be accessed and applied. IQATS first tries to pattern match the sentence, if successful, it will store the name of the pattern in memory to match with the patterns for the question and answer to be applied. By tracing each phrase-marker with their respective words this can easily be seen. This foundation easily gives ground to learning, as should be noted, there are certainly other ways to ask questions from the evidence given by "Bill Clinton was the President in 1993". If IQATS is unable to match the pattern of a specific sentence, or if the user feels another question can also be asked, the user can simply input an appropriate question and answer designated for the sentence. IQATS will learn the pattern and apply it to other sentences with that pattern. 5. Take the previous sentence. Assume that the user wishes that IQATS also ask 'When was Bill Clinton the President?' and answer 'in 1993'. IQATS will parse 'was Bill Clinton the President', leaving 'when' out, by checking for question words in apply.dat. Then it will retrieve the pattern: (verb-to-be-phrase person object), as for the answer: (time), and index them into its memory. IQATS will also check to see if it needs to change tense by comparing if the previous sentence's verbs were past-tense and if the question was not present-tense (question.dat). The next time the user executes IQATS, it will look for the pattern for that particular sentence and find those patterns, listed above (note: more than one question can be generated at a time, no one question overrides another), outputing the appropriate question and answer. The following examples consist of LISP/SCHEME data structures (lists). The embedded parsed sentences are followed by their respective question and answers.
The city Abu Dhabi is found in Saudi Arabia
(("Where" "is" "The" "city" "Abu" "Dhabi" "?") ("Found" "in" "Saudi" "Arabia") ("What" "is" "found" "in" "Saudi" "Arabia" "?") ("The" "city" "Abu" "Dhabi"))
(("Why" "did" "Andrew" "Jackson" "became" "the" "president" "?") ("Because" "of" "the" "election" "of" "1800"))
(("What" "are" "Anions" "?") ("Groups" "of" "ions" "with" "a" "negative" "charge") ("What" "are" "groups" "of" "ions" "with" "a" "negative" "charge" "?") ("Anions"))
(("When" "did" "The" "Civil" "War" "occur" "?") ("From" "1861" "to" "1865")) 1. IQATS does not yet distinguish patterns with duplicate phrase markers and will overwrite ones the last of any duplicate phrase marker, with the first. This will be immediately implemented upon the next release. 2. Removing 'Unknown markers from identify.dat may cause IQATS to crash. The reason is yet 'unknown' :)
|
