MELISA. An ontology-based agent for information retrieval in medicine.

Jose María Abasolo, Mario Gómez

Institut d’Investigaciò en Intel.ligència Artificial (IIIA)

{abasolo, mario}@iiia.csic.es

Abstract

This paper describes MELISA - MEdical Literature Search Agent – a prototype of an ontology-based information retrieval agent. We have designed a modular system that can be easily adapted to another medical literature sources or other professional domains. The major issues are the design of an architecture with three levels of abstraction, the use of separated ontologies and query models, and the definition of some aggregation operators to combine results from different queries.

Keywords: Knowledge-based mediation architectures

Introduction and motivation

In Internet, there are a lot of general-purpose search engines, witch goal is to retrieve web pages matching some criteria. In addition, there are also some professional engines which results are literature references: MedLine is a good example of this. It is a large database with biomedical bibliographic references, so we think it is a good starting point to develop an information retrieval agent for professional purposes: looking for literature references. In order to achieve this goal, we have carried out a process of knowledge analysis with a professional in medicine, and we have developed a prototype based on the resulting medical ontology.

Without detailed knowledge of the collection make-up nor of the retrieval environment, most users find it difficult to formulate well-designed queries for retrieval purposes. In fact, as observed with Web search engines, the users might need to spend large amounts of time reformulating their queries to accomplish effective retrieval. The user usually make a first query, sees if the information is retrieved and examine how useful it is for his need. Most of the times he has a large list of documents, intractable for him, other times he had restricted so much the query that the result is not sufficient. At this point he has to reformulate his query.

The purpose of this project is to solve this typical problem within a professional domain, in our case medical literature retrieval. To make this we implement different modules to generate queries, evaluate the results, reformulate the queries, if it is necessary, and show the results to the user.

In this first approach we work with the MedLine database to make the search, but keeping a structure that will allow us to work, in the future, with different databases or other search engines. Furthermore, to get a more practical realisation, we have adopted a well-known medical paradigm: "Evidence Based Medicine".

In section 1 we describe the system overview. Section 2 explains which type of information retrieval is needed to work with MedLine database, and MedLine itself. Section 3 describes the construction of a medical ontology. Next section addresses the design of the query models. Section 5, 6 and give a description of the generation and evaluation of the queries. Section 7 shows an example. Finally we present some conclusions and future work.

1. Overview

Here we present a general scheme of the prototype that we have designed to accomplish our objectives. We call it MELISA - MEdical LIterature Search Agent.

  1. Input interface: Allows the user to specify the main topics to perform the search, some typical medical categories and other search modifiers as date of publication or evidence quality. All this data constitutes a Consultation. We can consider a consultation as a very abstract - independent of the database-, conceptual, high level query.
  2. Query Generation & Reformulation: A consultation becomes the input for the Query Generation module. This module is the core of the entire system. It takes a Consultation, the Medical Ontology and the Query Models; so it can transform a Consultation into a collection of low-level, database dependent queries. We call them Specific Queries. As we are going to see later, there is another information level between Consultation and Specific Query; it is based in a domain knowledge model that we call Conceptual Query, as it acts as a link between the Consultation and the Specific Query levels. In addition, this module can reformulate the Specific Queries when the results are insufficient.
  3. Query Evaluation: The results of the Specific Queries - basically a collection of literature references - are then moved to the Query Evaluation module. The function of this module is to assign some score to these references, according to some criteria, as the degree they fulfil the user specifications or the evidence quality of the studies referred by the literature. Furthermore, the results for the Specific Queries are joined within a different group for each Conceptual Query.
  4. Filter & Combination: The literature references must be filtered and combined to reach a definitive score. This process is necessary due to the nature of the queries, which make possible repeated references. First, we need to eliminate different apparitions of the same reference; second, it is recommendable to delete references that do not fulfil a minimum constraint-satisfaction criterion.
  5. Output Interface: Last, the results are interactively recombined to be shown to the user in an appropriate manner.
  6. Query Model: It refers to information schemes that represent queries at various abstraction levels. We have mentioned two of these levels before Consultation –the higher -, and Specific Query, - the lowest -. It allows us to make the agent the more independent from the context, being the context a concrete search engine like PubMed.
  7. Medical Ontology: It contains some medical knowledge used to generate the queries. It was designed as a hierarchical tree, with a frame-based representation approach. This ontology must be at some degree context free, but it has to point elements of the search engines used by the Query Generation module.
  8. PubMed & MedLine: At this moment, we are working only with one database – MedLine - and its associated web-based search engine – PubMed -. But it is important to remark that our goal is to develop a general framework that allows us to reuse modules of this agent with other medical databases and also other professional domains.
  9. MeSH Browser: It is the access door to the MeSH Ontology, it allows to test the terms used as keywords and give the user some additional information about them.

2. Medline

In this section we explain the structure of MedLine’s documents, and how PubMed works. We need this knowledge because the entire project has been developed using the idea of working with metadata collections, more useful for professional purposes.

Metadata collections are sets of documents with additional information about the document, called metadata. Metadata is information on the organisation of the data, the various data domains, and the relations between them. Summarising, metadata is ‘data about the data’. Common forms of metadata associated with text include the author, the date of publication, the source of publication, the length, and the document genre. This kind of metadata is usually called descriptive metadata. Another type of metadata characterises the subject matter that can be found within the document’s contents. We refer this as semantic metadata.

MedLine Database is a metadata collection referred to biomedical articles. The articles stored in MedLine have both Descriptive and Semantic Metadata. We will see later the structure of MedLine’s documents.

To standardise semantic terms, many areas use specific ontologies, collection of concepts, terms and relations between them, used to describe the knowledge domain. In MedLine, this ontology is called MeSH (Medical Subject Headings).

2.1 MedLine’s documents

As we have said, MedLine’s documents have more information than the simple article reference. We work with these fields to make a query. The most important difference in MedLine is the MH field that gives us the meaning (Semantic Data) of an article.

2.2 MeSH Ontology

MeSH (Medical Subject Headings) is a medical ontology made with 18000 categories. The structure of MeSH is a polytree, a hierarchical structure where a term can appear in different branches. MeSH objects can be described with these properties:

MedLine has a set of subheadings, but each term has only a subset of allowed subheadings.

The MH field of a document contains a MeSH term and optionally some of the allowed subheadings. This field represents one of the topics of the medical reference. In addition, one MeSH term can be flagged as a Major Topic – the most representative -.

2.3 Search Modifiers

PubMed allows performing different types of searches for a keyword by using some constraints that we call search modifiers. Different search modifiers impose different constraints on the search, restricting the set of data fields used to carry out the search.

MAJR, MH:NOEXP and MH can be applied to MeSH terms. TI and TW can be applied to any word or expression. PT can only be applied to a set of allowed values.

As we will see later, our system uses these modifiers to perform multiple queries for a term, looking at the medical ontology to determine which type of modifiers are allowed. The evaluation procedure assigns scores to articles according to the search modifier used in the associated query.

3. Medical Ontology

Our goal is to develop a system able to work with diverse information sources. We think that ontologies are the best approach for information retrieval tasks in so heterogeneous environments, supporting both structured and unstructured data.

We have carried out a knowledge analysis process working along with an expert. The result of this process consists of a collection of medical concepts that the expert considers relevant in medical practice, and some search terms for these concepts. Then the system translates them into a conceptual category-based structure. The categories were iteratively refined and the search terms systematically tested to obtain good results.

There are two main desirable properties to have in mind when designing the medical ontology: It should be an appropriate knowledge representation of the world, and it must point to a good set of terms in the search environment. Another aspect of interest, is the role the ontology can play as a user guide to specify useful queries.

We use a frame-based approach. From this point of view, we represent the knowledge into a hierarchical tree, with classes, subclasses and instances of these classes. Each object has some slots or properties that we can define as a data type. The descendants of a class inherits its slots and values

We distinguish three abstraction levels.

The first level refers to main areas of medical issues. These groups serve us as an organisation scheme, an also, it allow us to apply different treatments to different category groups. For example, we can use different weights to score articles, according to the relative importance of the categories they belong to.

The medium level is constituted by a collection of medical categories. They are probably the most important concepts, as they have been chosen to capture issues of interest for the medicine professionals when looking for literature references in general, and from an EBM practical approach in particular. The data structures at this level have to point elements of the third level.

MeSH terms are our particular representation of terms existing in the MeSH Ontology (the ontology used by Medline its associated search engine, PubMed). We have developed a module to capture and represent these terms in our ontology.

Next, we present the first two levels of the ontology and a complete description of one medical category and its related MeSH terms in order to understand this knowledge representation

We have defined a class at the top of the tree, called Medical Class, and four subclasses to represent groups of categories: Evidence quality, clinical categories, analysis and evidence integration, each one with a little number of instances, one for each medical category.

Some slots are single data types, but there are another ones that are references to other objects of the ontology.

Every subclass of the Medical Class specifies a class name; the rest of the slots are inherited without value, so we do not represent these slots.

Each Medical Category was represented as an instance of one of the four groups of categories; for example, we define a category for medical guidelines, belonging to the group called Evidence- Integration.

At the lowest abstraction level, we define a structure called MESH_TERM, to represent the MeSH terms, but only those terms used by our particular medical ontology. Note that this structure is very similar to the one described in the MeSH ontology.

We define two objects as single data types – strings- to represent subheadings and publication types. But it is important to remember that these strings can be only allowed PubMed publication types and subheadings.

As other search engines may need some different data, we can adapt this ontology to suit new concepts, by modifying or adding new concepts.

4. Query model

To understand the following sections we give now some definitions of data structures that we call query models, because they represent information about queries. First of all, we consider three levels, as in the mesh ontology.

The first level – Consultation - groups all the information needed to perform a search.

The conceptual queries are directly associated with the medical categories in the ontology, one conceptual query is needed for each medical category included in a search.

Last, specific queries are suited to define and capture the results of the real, physical queries. Each one is the result of combining some keywords with a search term given by the ontology, plus some search modifier.

These models drive the construction of queries, as well as their evaluation. For this reason, each level contains a collection of structures of the level below. It allows the query generation module to go down during the generation of the queries, and reverse when evaluating and combining the results of the queries.

The idea of decomposing the query model in three abstraction levels is to facilitate reusability: working with other search engines and other domains. Only the lowest level depends on the specific search engine. The higher level is absolutely independent, and the medium level is dependent only because their association with the medical categories defined in the ontology.

4.1 Consultation

<

A consultation is the representation of the user’s need. The user gives these elements in the consultation window. Let us see which are these elements and their meaning.

The medical categories are selected from those categories defined in the medical ontology. The list of conceptual queries is made by the Agent, during the query generation procedure. The other fields are given by the user.

4.1.1 Keywords

The keywords are the core of the search process. They are words or expressions representing the main topics to drive the search. Any string is allowed as a keyword, but it is very recommendable to use valid MeSH terms as keywords, so the search may obtain better results, and the user should apply some subheadings to restrict the search. These keywords are represented in a similar manner as the MeSH Terms, plus some additional information:

Note that Description, Fathers, Sons, Subh, Subhsel and RelKW are only available if the keyword is a valid MeSH term. All this information is used later to generate and reformulate the queries.

4.1.2 Medical Categories

<

The user can select some categories from the Medical Categories described at the previous section, which allows the user to focus the search to their main interest areas. All this categories may be selected with independence of the others, but those categories about the quality of the evidence. In the last case, the user should only choose the minimum quality degree of the evidence.

4.1.3 Special Filters

This is not medical information, but descriptive information. There are three filters:

- Year_From: This filter take out of the search documents published before this year
- Year_To: Take out documents published after this year.
- Abstract: When true, the search process retrieves only documents with abstract.

4.2 Conceptual Queries

A conceptual query is a structure between consultation and specific queries. It not depends on the search engine. A conceptual query has these elements:

4.3 Specific Queries

This is the low-level structure. It is really the one that works with a search engine, so it depends on the search engine.

5. Generation and reformulation of queries

Here we describe the process of transforming a consultation into a collection of specific queries, and how these queries are sent to the search engine. Before explaining this process in detail, let us discuss different approaches to the query generation task.

As we have said before, a consultation implies some conceptual queries, one for each selected medical category. Furthermore, we know that a medical category links to a collection of MeSH and non-MeSH terms, plus other concepts like publication types. In addition, each term can be appended with some search modifiers. Thus, our system has to address a large amount of information. Therefore, we must design a method to minimise time cost and maximise information quality.

We have considered three aspects to design the query generation strategy:

- To send all queries in parallel, so we avoid waiting for one query to send another.
- To use a short retrieval format, during the generation and the evaluation procedure we retrieve only the document references (UID’s) to economise time and space resources.
- To perform short queries – with a few number of search terms- rather than long queries. This allows working only with UID’s, as the evaluation procedure can score documents according to the queries they appear in.

5.1 Decomposition process

We can see the process as an iterative decomposition process, from the most abstract level – consultation - until the lowest level - specific queries. As we will see later, the evaluation and combination process can be considered the inverse, as the systems proceeds by integrating results from different queries into more general objects.

The first step consists on the generation of the conceptual queries, one for every medical category selected by the user, and another extra conceptual query to perform searches with only the keywords and no medical category. Each conceptual query takes three parameters as inputs: the list of keywords, a medical category, and the special filters.

In the second step, the query generator executes a decomposition of each conceptual query into a set of specific queries. Each specific query is constructed by combining the keywords and special filters of one conceptual query with the items that represents the medical category for that conceptual query and its allowed search modifiers. Every conceptual query can result in a very different number and type of specific queries, according to the structure of the medical category associated with that conceptual query.

Keywords, special filters and terms related to the medical category are combined with the AND operator - the most restrictive one. After the evaluation process, if the results are not sufficiently good the generation process produces a new collection of specific queries, but now using the OR operator. At the moment we use only a quantitative criterion, in short, reformulate if the number of retrieved documents is less than a fixed threshold.

6. Combination and evaluation of queries’ results

As we have seen in the previous section, we generate a collection of specific queries for each conceptual query. The specific query interacts with the search engine – or engines – and retrieves a list of documents. In this section we explain how to join all the documents corresponding to a conceptual query, plus the functions used to score documents.

Scoring documents inside a conceptual query can be seen as assigning documents with a membership value, referred to the medical category associated with that conceptual query. So, the key is to understand the meaning of the search concepts defined in the ontology, as well as the sense of the search modifiers that can be applied to these concepts. Thus, we distinguish some order relations that can be used with a membership meaning.

a) Search concepts (specific query’s Type):

Mesh_Terms>Related_Mesh_Terms>NoMesh_Terms
Subheading > Subheading2
Publication Type

b) Search modifiers (specific query’s Subtype):

MAJR> MH:NOEXP > MH > TI > TW > no-modifier

To score documents, we define a data structure with a numeric field for each search concept (Mesh_Score, Mesh2_Score, NoMesh_Score, Subh_Score, Subh2_Score, PubType_Score), and a field to store the overall score (Global_Score) of a document in the conceptual query.

This scoring process has two steps: First, MELISA has to score the documents inside a conceptual query. Second, MELISA has to combine the results of different conceptual queries, according to the user’s selection.

6.1 Scoring documents inside a Conceptual Query

After retrieving, MELISA has the following information: lists of articles – only UID – retrieved in the different specific queries. Now the agent needs to join all these lists into one list with some order. To achieve this goal the system uses an aggregation function that enables us to join the information of all the specific queries inside a conceptual query. MELISA uses an aggregation function to sort the resultant list.

6.1.1Aggregation function

At this point we need to calculate the importance of a document (a) inside the conceptual query j. This importance depends on the occurrences of the document in the different specific queries results. The aggregation function is defined as follows for a conceptual query j:

[1]

Where a is the retrieved document, ci is the weight coefficient for a search modifier i. These coefficients have the following restrictions:

;

And q ij is a membership function of documents obtained with the search modifier i in the conceptual query j.

6.1.2 Membership function

[2]

Where nijk(a) is the number of occurrences of the document a in specific queries with search modifier i and search concept k, in the conceptual query j. Nij is a normalisation coefficient defined as follows:

[3]

Nijk is the number of specific queries with search modifier i and search concept k, generated for the conceptual query j. It represents the theoretic maximum for nijk .

The weight coefficients bk for a search concept k, must accomplish the following constraint:

6.1.3 Weight coefficients

We have two sets of weight coefficients for the search modifiers, depending on whether or not the conceptual query has a publication type (PT).

Ci

Conceptual query without PT Conceptual query with PT

MAJR

0.3

0.15

MH:NOEXP

0.25

0.125

MH

0.2

0.1

TI

0.13

0.065

TW

0.08

0.04

PT

 

0.5

Without Search Modifier

0.04

0.02

Furthermore we have another set of weight coefficient for the search concepts:

bk

 

Mesh terms

1

Related Mesh terms

0.7

Non-Mesh terms

0.5

Subheading

1

Related Subheading

0.7

Publication type

1

These values have been estimated empirically. In the current implementation of MELISA, the weight coefficients are static values, but in the future we plan the agent to learn these coefficients by interacting with the user.

6.2 Combination of documents from different conceptual queries

At this point, the system has a list of scored documents for each conceptual query. MELISA allows the user to combine the results of different conceptual queries. For this purpose, we need to compare and combine the score of articles in different conceptual queries.

This is not an easy task, because the conceptual queries are associated with a medical category, and these medical categories have not the same structure. This structure depends on the number of terms, used to define a medical category, the specificity of these terms and their type. A consequence of this is that not all the documents can easily get high scores, it depends on the conceptual query where they have been retrieved. It is not the same a best document for a conceptual query with score 0.8 than a best document with score 0.2

We want a function that applies different corrections in the document scores according to the empirical maximum reached. For the categories obtaining a maximum score of one, no modification is needed, for the categories with a lower maximum; we apply a greater correction to the score. For example, let us suppose we have a document with the following scores.

 

Score

Maximum

Desired Increment

Category 1

0.3

0.5

Medium

Category 2

0.1

0.1

Large

Category 3

0.8

1

Small

To achieve our goal, we define the following function:

[ 4]

Where N is the number of conceptual queries we want to combine and Kj is the maximum of Q j(a).

With these new scores of the documents, MELISA sorts the resultant list of documents that will be shown to the user.

7. Example

Here we present an example extracted from the real medical practice.

Let us suppose that the user looks for information on current Levofloxacin treatments of the pneumonia. In addition he wants to know if there is some evidence concerning this (EBM), thus he should create the next consultation:

Keywords: ‘Levofloxacin’, ‘Pneumonia’

Medical_categories: ‘Good evidence quality’, ‘Therapy’, ‘Recommendations based on the evidence’, ‘Guidelines’, ‘Cost Analysis’

Special filters:  Documents accepted from 1960 to 2000,  Only retrieve articles with abstract

Figure 4 shows the Consultation window for this example. At this window the user can add or delete keywords, select medical categories and set special filters.

Each new keyword is automatically checked against the MeSH ontology and the list of allowed subheadings is shown if they are valid mesh terms. Information about keywords is shown in the MeSH Window. Figure 5 shows the MeSH window that corresponds to the term ‘Levofloxacin’. It is not an exact MeSH term, but is associated with the MeSH term ‘Ofloxacin’. At this point the user has two options, to accept the MeSH term given by the MeSH Browser or refuse this term and try with another one.

At this window, the user can also specify some subheadings for the keyword.

If the user puts a term that is not a valid MeSH term, the system advises him of this condition, and shows a list of similar valid MeSH terms. Then the user can choose one of these suggested terms, nevertheless he can use his own, not Mesh term.

Figure 6 shows an example of this case. We have simulated a human mistake, i.e. the user writes ‘Levoflaxin’ in place of ‘Levofloxacin’. The system shows a list of terms, starting with ‘Levofloxacin’, so the user can choose the correct one.

After the first decomposition task, we obtain 6 conceptual queries, one for each selected medical category, and an additional conceptual query with only the keywords.

1. Ofloxacin + Pneumonia + GOOD_EVIDENCE_QUALITY
2. Ofloxacin + Pneumonia + THERAPY
3. Ofloxacin + Pneumonia + EBM
4. Ofloxacin + Pneumonia + GUIDELINES
5. Ofloxacin + Pneumonia + COST_ANALYSIS
6. Ofloxacin + Pneumonia

Now, we will see in detail the conceptual query generated for the medical category ‘Good Evidence Quality’. First, we show the structure of this instance.

The second decomposition procedure generates 6 specific queries for every MeSH term in the conceptual query associated with ‘Good Evidence Quality’, i.e. with the term ‘Meta-Analysis’

1. Ofloxacin * Pneumonia AND Meta-Analysis[MAJR]
2. Ofloxacin * Pneumonia AND Meta -Analysis[MH:NOEXP]
3. Ofloxacin * Pneumonia AND Meta -Analysis[MH]
4. Ofloxacin * Pneumonia AND Meta -Analysis[TI]
5. Ofloxacin * Pneumonia AND Meta -Analysis[TW]
6. Ofloxacin * Pneumonia AND Meta -Analysis

As these medical category has four MeSH terms, it produces 6 * 4 = 24 queries.

One specific query is generated for every publication type

1. Ofloxacin * Pneumonia AND Meta-Analysis[PT] 2. Ofloxacin * Pneumonia AND "Randomised Controlled Trial"[PT] 3. Ofloxacin * Pneumonia AND "Clinical Trial, Phase III" [PT] 4. Ofloxacin * Pneumonia AND "Clinical Trial, Phase IV" [PT]

In total, 28 specific queries are generated for the first conceptual query. Note that the results of each specific query must be combined later into a single collection of documents, evaluated and sorted by means of an aggregation function.

The process is repeated for the remaining conceptual queries. There is a special case, the conceptual query with no medical category. In this case, the system has to combine only the keywords with their allowed modifiers and any search term from the medical categories.

1. Ofloxacin[MAJR] * Pneumonia[MAJR] 2. Ofloxacin[MH:NOEXP] * Pneumonia[MH:NOEXP] 3. Ofloxacin[MH] * Pneumonia[MH] 4. Ofloxacin[TI] * Pneumonia[TI] 5. Ofloxacin[TW] * Pneumonia[TW] Ofloxacin * Pneumonia

Where * represents an operator for combining keywords {AND, OR}.

After scoring all the documents in all the queries, the system shows the documents better fulfilling the user requirements. The Results window (Figure 7) consists basically of a list of the documents retrieved, ordered according to the combined score. The user can select some of the categories used to perform the search in order to combine all their documents in only one list. Note that only the basic information about the documents is shown at this window: authors, title, publication data and source of publication. We use a hypertext format to easily focus on any document. The authors’ information is a link to the extended information about an article, showed in the Document viewer (Figure 8). Furthermore, we insert two special links: one to append a document to a list of the documents he wants to order; another to show related articles suggested by MedLine. Figure 7 shows documents corresponding to the default set up; these are the best documents resulting of combining all the categories selected in the consultation window. Article.

Remember that MELISA only works with document’s UID (unique identifier), thus the system has to retrieve the complete information about the articles before to display them. Since this is a very cost expensive process, the system shows only twenty documents by page

The Document viewer extends the information offered in the Results window, including the abstract, the mesh terms and the major topics for an article, plus the scores for all the categories and the combined scored. Figure 8 shows the Document viewer when the first document in the Results window is selected.

8. Conclusions and future work

Before addressing evaluation of performance and future work, let us discuss the reasons to adopt this approximation to the evaluation procedure instead of those typically used in IR

Typical IR functions compare documents to queries based on term frequencies. They are best suited to work with unstructured data, but can also be used to deal with structured data, as in MedLine. The problem is that the system needs to retrieve, store and analyse a big amount of information, so it may be unpractical or too much resource consuming. We present a method to avoid this problem: instead of retrieve a document only one time with a complete description to score it, our system generates a big quantity of different queries for each concept, retrieving only the identification number of the documents in the database. Our scoring procedure is based in the quantity and characteristics of the queries where documents appear. We think this approach take benefit of the Internet properties, since it is amenable to parallelize the execution of the information retrieval task.

Currently we are evaluating the system performance by means of comparing the results offered by our system against the results obtained by humans using the MedLine web-based searcher: PubMed. An expert has proposed us a collection of real medical cases. Then we have translated these cases into MELISA’s consultations. We have tested two different evaluation functions:

  1. Multi-valued logic using t-norm operators.
  2. Aggregation operators

An expert has evaluated the results offered by our system applying a classification of the documents retrieved by MELISA. He can assign a document to one of these three categories:

  1. The document do not satisfies the user requirements
  2. The document is related with the requirements but do not satisfies them enough
  3. The document satisfies enough the user requirements

We compare the frequencies of each group to determine the relative performance of the evaluation functions. We will present a systematic analysis of the results in future papers; nevertheless, we present some conclusions about the comparison of both methods.

The approach based on multi-valued logic seems to be better to combine documents between conceptual queries. The problem is on the loss of information when calculating scores within a conceptual query.

The finally adopted approach - aggregation operators - is very accurate when evaluating documents within a conceptual query. Its weak point lies on combining documents from different categories, because every category has a different statistical distribution of the scores. The evaluation procedure confirms our analysis; we detect a loss of precision when incrementing the number of categories combined to obtain the overall result.

MELISA is able of processing, scoring and combining a large amount of medical literature in an acceptable way, avoiding the user of a tedious and imprecise work. However, we plan to do some work to improve and extend the capabilities of our information agent in the future.

First, we will develop user profiles, in order to develop a system able to adapt to different users. Second, we will add some capabilities to allow the system to work with other medical databases available on-line. Third, we should add capabilities to handle different evaluation functions, as fuzzy measures and WOWA operators. Fourth, we must study more complex criteria to determine when reformulating the specific queries. Five, we will compare the different evaluation functions and combination of these functions. As we can distinguish evaluation within a category from evaluation between categories, we can apply different methods to both separated functions. Finally, we think that it is very interesting to incorporate methods to learn the weight coefficients and the user profile.

Acknowledgements

The authors would like to thank the Spanish Scientific Research Council for their support. This work has been developed under the SMASH project (TIC96-1038-C04-01) and the IBROW project (IST-1999-190005).

Special thanks to Enric Plaza for their suggestions and Albert Verdaguer for his assistance in the analysis of the medical domain.

9. References

  1. Arens, Y., Knoblock, C.A. & Shen, W.M. Query Reformulation for Distributed Information Gathering
  2. Baeza-Yates, R. & Ribeiro-Netop, B. Modern Information Retrieval
  3. Boyan, J., Freitag, D. & Joachims, T. A Machine Learning Architecture for Optimising Web Search Engines
  4. Cardelli, L. & Davies, R. Service Combinators for Web Computing. SRC Research Report, June 1, 1997
  5. Chen, C. Structuring and Visualising the WWW by Generalised Similarity Analysis
  6. Chen, C. & Czerwinski, M. From Latent Semantics to Spatial Hypertext –An Integrated Approach
  7. Edwards, P., Bayer, D., Green, C.L. & Payne, T.R. Experience with Learning Agents which Manage Internet-Based Information
  8. Feinstein, A.R. & Horwitz, R.I. Problems in the "Evidence" of "Evidence-based Medicine"
  9. Lawrence, S. & Giles, C.L. Context and Page Analysis for Improved Web Search
  10. Mahalingam, K. & Huhns, M.N. An Ontology Tool for Query Formulation in an Agent-Based Context
  11. Montbriand, J. Extending and Controlling Sherlock and the Find by Content Libraries. Apple Developers Technote 1141
  12. Oates, T., Nagendra Prasad, M.V., Lesser, V.R. & Decker, K. A Distributed Problem Solving Approach to Cooperative Information Gathering
  13. Payne, T.R. & Edwards, P. Learning Mechanisms for Information Filtering Agents
  14. Pratt, W. Dynamic Organization of Search Results Using the UMLS. Proceedings of the American Medical Informatics (AMIA) Fall Symposium (Formerly SCAMC), 1997
  15. Pratt, W. Physician’s Information Customizer (PIC): Using a Shareable User Model to Filter the Medical Literature. Proceedings of the International Conference on Medical Informatics (MEDINFO95), July 1995