|
|
CXP-Powered I-MINDS Capabilities 3.0 |
QuestionAnswering
3.0
This QuestionAnswering
capability was completed in December 2005 and revised in May 2006, and revised
again in December 2006. This capability
has two views: an instructor application and a student application. The person in charge of the Q&A session
(e.g., an instructor) can pick and choose questions to answer. The application helps manage the
questions. The users asking questions
(e.g., a student) can review their questions and answers. They can also ask
questions.
This capability
also has agent intelligence. The
instructor application scores each question based on keywords and also “machine
learns” from the instructor about what keywords are important
behind-the-scenes, without interfering with the instructor’s actions. This allows the capability to learn to score
each question more accurately over time.
As this is one
of our intelligent capabilities (which are usually significantly complex than
our other capabilities), here we provide a detailed discussion of the features
and requirements, including installation guideline.
1. Capability
1.1.
Introduction
As with previous
versions, the question answering mode in version 3 retains both the student and
teacher version. Both versions were
originally based on the chat capability that comes with ConferenceXP. The teacher version was heavily modified to
enable lists of previously received questions to be displayed and sorted. Based on feedback from a test run by
1.2.
Student Flow of Control
Students using
the question answering mode can ask the teacher a question by entering the
question into the box at the bottom of the Student form (see figure 3.1) and
pressing the Send button. The question
is then encapsulated using a message object with the QA_Question message type
by the buddy manager agent. The
resulting message object is then multicast using the SendObject method to the
rest of the classroom. In affect, the
student version broadcasts the question to all the other instances of the
Question Answering capability (both teacher and student) running on other
machines. The teacher version will be
listening for such questions.
Figure 3.1:
Student Version of Q&A.
Figure 3.2:
Teacher Version of Q&A
The student
version also listens for responses sent by the teacher version. Whenever a message with the QA_Response
message type is received, the sender identifier is checked against the
identifier for the local participant. If
a match is found then the response is being sent to this student version. The actual response content is displayed
along with the original question in the ‘Recent Answer’ and ‘Recent Question’
boxes. They are also added to the ‘Prior
Messages’ table. Figure 3.1 shows how
the student version looks after it has received a response. Students can resend questions to the teacher
by right-clicking on the question or answer and pressing ‘Resend’ on the popup
menu.
1.3.
Teacher Flow of Control
Teachers using
the question answering mode will see the form shown in figure 3.2. The teacher form displays all the questions
that have been previously asked by students in the classroom. Questions that have already been answered or
discarded will appear in the appropriate tabs.
Questions that are still waiting for a response will appear in the ‘Received
Questions’ tab, sorted by score. When
the teacher right-clicks on a message in the ‘Received Questions’ table he or
she has the option of either answering or discarding the question. If the teacher selects the question, it will
appear in the ‘Current Question’ box.
The teacher can then write a response to this question and press
‘Answer’ button. A new message with the
QA_Response message type is then created with the response and the message
object is then multicast using the SendObject method to the rest of the
classroom. When the teacher answers or
discards the question, the question answering agent will modify the weights of
the keywords it contains. See the next section
for more details.
In version 3, we
have added the ability for the teacher to send message to the students rather
than just waiting to respond to their questions. We do this by maintaining a list of students
who are currently in the classroom and running the Q&A mode. By selecting individual students or selecting
the ‘To All’ checkbox the teacher can determine which students will receive the
message. When the teacher presses the
‘Message’ button, whatever she has written is encapsulated into a message
object with the QA_Message message type and sent to all the students who are
selected. The students can easily
distinguish between messages and responses from the teacher because the Q&A
agent indicates this on the student form.
As mentioned
above, the teacher version listens for questions asked by students. When such a question is received the question
answering agent first determines what score the question should receive. This is done by first parsing and tokenizing
each word in the question. For more
information please consult the Question Answering Agent section below. The question immediately appears in the
‘Received Questions’ table and is available to the teacher. As this table is sorted by score, good
questions will appear higher on the table while bad questions might not even
appear on the table unless the teacher scrolls down.
1.4. Intelligent
Agent
The teacher
version of the question answering mode contains an intelligent agent whose
purpose is to aid the teacher in deciding which questions to answer. It does this by assigning a score to each
question as it is received from the student.
This score is determined by the length of the question and the weights
of the keywords it contains. When a
question is first received it is tokenized and stemmed by a natural language
processing program. The stems are checked
against the keywords stored in the database.
If a match is found the weight of the keyword is added to a running
total. This total is then normalized
along with the length of the question to determine the score.
The intelligent
agent also adjusts the keyword weights up or down. This is done when the teacher decides to
answer or discard a question. When the
teacher answers a question the question is once against tokenized and
stemmed. The weights for keywords found
in the question are increased. Stems
that are not found in the keywords table are added as keywords with a low
weight. When the teacher discarded a
question the weights for the keywords found in that question are decreased.
The natural
language processing program we are using is called “combiner.exe”. It maintains a taboo list of certain common
words. All keywords are automatically
normalized by the intelligent agent to fit a certain range. This prevents the weight associated with a
common keyword from getting so large that it dominates the rest.
In pervious
versions the machine learning weights were hardcoded for the intelligent
agent. There was also only one set of
keyword weights, hardly suitable for a system that should support multiple
teachers and classrooms. In version 3,
we have added the machine learning weights to the database so they can be
adjusted without recompiling. Each
classroom has its own set of machine learning weights that the teacher can
modify and the keyword weights are now specific to each teacher and classroom
combination. This allows the I-MINDS
system to store the same keyword with different weights for two different
classes taught by the same teacher.
1.5. Adding
Keywords
The teacher can
at any time select the ‘Add Keyword’ item from the options menu. This brings up a form containing all the
keywords and weights stored in the database for that teacher and classroom
combination. Unlike in previous
versions, keywords can no longer be deleted by using this form. New keywords can be added by clicking on the
star row at the bottom. Existing
keywords can be updated by selecting their entries and typing in a new
weight. To save the changes simply press
the Update button. An example of this
form is shown in figure 3.3.
Figure 3.3:
Add Keyword Form
2. Teacher Options
To further
assist the teacher we have developed the Virtual Classroom and Statistics
Charts modules. These modules can be
started from the Options menu.
2.1.
Virtual Classroom
The Virtual
Classroom form is very useful for teachers.
It displays information about each student, message, and topic from a
classroom. Students are represented by
small pie charts, with each chart split into five pieces. Each piece corresponds to a different
statistics category. The pieces are
colored blue if the student is at or above the thresholds for those statistics
(as set by the teacher) and colored red if the student falls below the set
thresholds. These thresholds are stored
in the database so the teachers can just “set it and forget it”, rather than
worry about their levels. The statistics
are computed for the last number of sessions specified at the bottom left
corner of the screen. An example of the
Virtual Classroom for can be seen in figure 4.1.
The messages and
topics for the classroom are displayed in list views on the right hand side of
the screen. All information tracked
about each message/topic by I-Minds is displayed in these panes. Also, just like in Database Search, a message
can be right clicked on, bringing up a “Message Tree” option. Choosing this option causes a tree describing
the thread containing that message/topic to be drawn for the user to work with.
Finally, this
form can also be used to find out specific information about individual
students and buddy groups in the classroom.
A student’s pie chart can be right clicked to display all the messages
by just that one student, or to find out information about their buddy
groups. If the “Buddies” option is
chosen, just that student’s buddies are displayed along with their messages and
topics. Additionally, there are check
boxes for each student which can be checked to get overlapping buddy
groups. For example, say a teacher
wanted to find out information about all the buddies that both student A and
student B share. Their check boxes can
be selected and then the teacher can click on the Buddies button at the bottom
of the form to display information about the overlapping buddies.
Figure 4.1: A Virtual Classroom Form displaying info
about two students
2.3.
Statistics Charts
The Statistics
Charts form is used to display statistics information about students in
graphical form. A teacher can choose to
display all the statistics of any individual student or the ratings of each
student for an individual statistic, both in bar chart form. Once again, statistics that fall at or above
the thresholds set by teachers are shown in a blue color and those that fall
below thresholds are shown as a red color.
The thresholds are the same ones recorded in the database by the Virtual
Classroom form, and any change to the thresholds is propagated in both
forms. There is also the option of
displaying a legend for the statistics by clicking on the Legend button. Once again, the stats are computed for the
number of sessions specified in the bottom left corner of the screen, which is
also tied to the corresponding option in Virtual Classroom. For an example of the Statistics Charts form,
consult figure 4.2.
Figure 2.1: A Virtual Classroom Form displaying info
about two students
Figure 2.2: A Statistics Charts Form displaying a
student’s stats in graphical form
3. Database
The I-MINDS
database is currently running MySQL version 5.
In the future we intend to switch over from MySQL to SQL Server or
something similar. The basic structure
should easily transfer over to any relational database. The database was designed using a combination
of DBDesigner4, an open source designer from fabForce, and phpMyAdmin-2.6.3. For an illustration of the I-MINDS database
consult figure 5.1 below. The mysqldump
textfile of the database can be found in the same folder as the source
code.
Unlike in
previous versions, in version 3 we have completely done away with the
ComponentTestLibarary.dll which contained DataAdapters. The merged capability class now has its own
database connection object, which is set to look at the DSN described in the
section 6 below. The merged capability
agent class now contains almost all of the database queries necessary to run
both merged capability and the buddy group and question answering modes. Certain peripheral classes such as the
MessageTree still maintain their own data tables, but use the database
connections stored in merged capability.
Figure 5.1:
The DBDesigner model for the I-MINDS database
As you can see
from figure 5.1, the database for I-MINDS version 3 has become fairly
complicated. It has grown considerably
from the one used in previous versions.
New tables were added to support the concept of classrooms. As a matter of fact, the classrooms table is
now the heart of the database replacing the sessions table from version 2. We have retained the message table but added
several new variables to support future work on agents. Several new tables have been added to support
the forthcoming structured group mode and the new buddy manager form in the
buddy group mode. There are several
satellite tables used for recording statistics on PGM experiments. The use of these tables will be addressed in
future technical reports.
6. Installation
This section
leads you through the installation for ConferenceXP and our capabilities. The first program to install is the
mysql-connector-odbc-3.51 driver. We
have included this with our source code.
Next, if you have not already done so download the CXP Client version
3.5. Before you install it make sure
that the Microsoft Journal Viewer has been installed. Please consult the ConferenceXP client
installation guide for more information.
We provide the
database dump file so that users can upload the I-MINDS database to their local
server. The installation is relatively
straight forward assuming you have access to MySQL server version 4.1 or
higher. Table 6.1 gives the console
command. If your machines do not have
access to the internet you will need to install your own venue service on one
of your machines. The Venue Service can
be downloaded from the ConferenceXP website.
mysql –uusername –ppassword <
iminds.sql
Table 6.1:
console command to upload database
Alternatively,
you could connect to the I-MINDS database running on cse.unl.edu. The user name and password are given in Table
6.2. A word of caution though, the I-MINDS database on CSE is our test bed and
is subject to design revisions which could break previous versions of the
capabilities.
Site: cse.unl.edu; Database: lmille;
MySQL user: lmille; MySQL password: GNEZWS
Table 6.2: connection
information for the I-MINDS database on CSE
To run the
capability you must first create an ODBC data source in Windows XP. This data source is what the C# code will use
to connect to the database. To do this,
open Administrative Tools/Data Sources (ODBC).
Add a new System DSN with the above information (see figure 6.1). If you are running your own MySQL server the
Server, User and Password entries may change from figure 6.1.
Figure 6.1: Data Source for
the I-MINDS database.
The merged
capability should be pasted into the CXPClient folder in the manner of all
capabilities. The Stemmer folder
containing combiner.exe and the licenseI-MINDS rtf file should each be pasted
into the CXPClient folder. There is also
an images folder used by the virtual classroom which should be pasted into the
same location. Remember that there are
two separate versions of the merged capability (one for teacher, another for
student).