Text retrieval systems - 4A: the Library

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • JosAH
    Recognized Expert MVP
    • Mar 2007
    • 11453

    #1

    Text retrieval systems - 4A: the Library

    Greetings,

    the last two article parts described the design and implementation of the
    text Processor which spoonfeeds paragraphs of text to the LibraryBuilder.
    The latter object organizes, cleans up and stores the text being fed to it.
    Finally the LibrayBuilder is able to produce a Library which is the topic
    of this part of the article.

    Introduction

    A Library manages groups of books. Each book contains chapters which contain
    paragraphs of text. The object can store itself in a compressed way given
    any OutputStream. A static method can load Library objects given any Input-
    Stream again. A Library can produce pieces of text, either entire books or
    chapters or a single paragraph of text.

    A Library can also produce lists of 'meta' data, i.e. it can produces a list
    of group names of book or chapter titles; it can also produce simple statistics
    such as the number of books in the library etc.

    Basically a Library can produce Lists. The Lists are either lists of pure
    text (Strings) or Lists of BookMarks. A BookMark represents a single paragraph
    of text and has additional functionality as well.

    Let's see how it's all done.

    The Library object

    In the previous part in the article we already saw a Library constructor being
    invoked by the LibraryBuikder; here are the data parts and de constructor of
    the Library object:

    [code=java]
    protected String title;
    protected WordMap wordMap;
    protected Section[] groups;
    protected Section[] books;
    protected Section[] chapters;
    protected String[] paragraphs;
    protected String[] words;
    protected String[] punctuation;

    protected NotesMap notesMap;

    public Library(String title,
    WordMap wordMap,
    Section[] groups,
    Section[] books,
    Section[] chapters,
    String[] paragraphs,
    String[] words,
    String[] punctuation) {


    this.title = title;
    this.wordMap = wordMap;

    this.groups = groups;
    this.books = books;
    this.chapters = chapters;
    this.paragraphs = paragraphs;
    this.words = words;
    this.punctuatio n= punctuation;

    this.notesMap = new NotesMap();
    }
    [/code]

    This is quite a bit of code but it is just simple stuff; all the parameters of
    the constructor are stored in the member variables. As already discussed, there
    are Sections for groups, books and chapters. The paragraphs and punctuations
    are strings. There's a A String[] array for the unique words and there's the
    WordMap that glues it all together. See the previous article part for an
    explanation of these arrays and objects.

    Also a NotesMap is initialized. a NoteMap is used for additional text for every
    single paragraph in the text. Here's its definition:

    [code=java]
    class NotesMap extends HashMap<Integer , String> {

    private static final long serialVersionUI D = 462767443691914 2999L;

    private boolean modified= false;

    public void setModified(boo lean modified) {this.modified= modified; }
    public boolean isModified() { return modified; }

    public void put(int paragraph, String note) {

    String oldNote= get(paragraph);

    note= (note != null)?note.trim ():"";
    note= (note.length() == 0)?null:note;

    if (note == null)
    modified|= remove(paragrap h) != null;
    else {
    put(paragraph, note);
    modified|= !note.equals(ol dNote);
    }
    }
    }
    [/code]

    Basically a NotesMap is a simple Map that maps Integers (the index number of
    a paragraph) to a simple String that stores the annotations for the paragraph.

    It applies a bit of intelligence to check whether or not the map was modified.
    Empty text is considered no text at all. The modified flag can be used to check
    whether or not the Library object should be saved again when it isn't needed
    anymore.

    The Library object implements a little convenience method so you never have
    to deal with a NoteMap object yourself directly:

    [code=java]
    public boolean isModified() { return notesMap.isModi fied(); }
    [/code]

    The NotesMap is not a very complicated nor a very interesting class except for
    one aspect: it implements the Serializable interface because its parent class
    implements this interface. This is needed because this class must be saved or
    loaded along with the Library object itself.

    As we'll see later the Library class itself also implements the Serializable
    interface.

    BookMarks

    As briefly mentioned in the introduction, a Library uses BookMarks to represent
    one single paragraph of text. Here's its interface definition:

    [code=java]
    public interface BookMark {

    public String getGroup();
    public String getBook();
    public String getChapter();
    public String getParagraph();

    public int getGroupNumber( );
    public int getBookNumber() ;
    public int getChapterNumbe r();
    public int getParagraphNum ber();

    public int getRelativeBook ();
    public int getRelativeChap ter();
    public int getRelativePara graph();

    public void putNote(String note);
    public String getNote();
    }
    [/code]

    If the Library has produced a BookMark object for you, you can query that object
    for several aspects of that text, i.e. you can get the name of the chapter the
    paragraph is part of; you can also get the name of the book where the paragraph
    can be found and you can find the name of the group where the book is stored.
    You can also get the text of the paragraph of course.

    Paragraphs, chapters, books and groups all have a number, starting at zero.
    There are methods available to retrieve those absolute index numbers. There
    are also methods available to retrieve relative numbers for those entities.
    Suppose there are two chapters in the first book and three chapters in a
    second book. The absolute chapter numbers are 0, 1, 2, 3, 4 and the relative
    chapter numbers are 0, 1, 0, 1, 2, i.e. the relative chapter numbers start
    at zero per book. A similar reasoning applies to books and paragraphs as well.

    Note that for obvious reasons there is no relative group number available.

    The last two methods set and get optional additional text given the paragraph
    represented by this bookmark. These two methods together with the Library
    delegator isModified() method completely shield the NotesMap class from you,
    i.e. you never have to deal with it yourself.

    Loading and Saving a Library

    If you want to load a library given an InputStream you don't have that object
    available yet (it isn't loaded yet). That's why the Library class implements
    two static convenience methods: one method can load a Library given a file name,
    the other method takes an InputStream from which it loads a Library object.

    Here they are:

    [code=java]
    public static Library read(String name)
    throws IOException, ClassNotFoundEx ception {

    FileInputStream fis= null;

    try {
    return read(fis= new FileInputStream (name));
    }
    finally {
    try { fis.close(); } catch (IOException ioe) { }
    }
    }

    public static Library read(InputStrea m is)
    throws IOException, ClassNotFoundEx ception {

    return (Library)new ObjectInputStre am(new InflaterInputSt ream(is)).
    readObject();
    }
    [/code]

    The first method just tries to open a FileInputStream given a file name and
    delegates the hard part to the second method. The second method wraps two
    other InputStreams around the given InputStream. The first wrapper/decorator
    inflates the input stream. The inflated input is used by the ObjectInputStre am
    wrapper that does the actual object construction. Both methods can throw simple
    IOExceptios when some reading fails and the can throw ClassNotFoundEx ceptions
    if the InputStream didn't make any sense.

    The second method doesn't close the stream passed to it because it doesn't own
    it, i.e. maybe the caller of this method has different plans for this stream,
    and closing it will prohibit it. The first method had created that OutputStrem
    itself, so it owns it and so it closes the stream itself.

    Of course you can only save a Library object if you have one. Therefore the
    write methods are not static methods. The logic of those methods don't differ
    much from the static load methods. Here they are:

    [code=java]
    public void write(String name) throws IOException {

    FileOutputStrea m fos= null;

    try {
    fos= new FileOutputStrea m(name);
    write(fos);
    }
    finally {
    try { fos.close(); } catch (IOException ioe) { }
    }
    }

    public void write(OutputStr eam os) throws IOException {

    DeflaterOutputS tream dos;
    ObjectOutputStr eam oos= new ObjectOutputStr eam(dos= new DeflaterOutputS tream(os));

    oos.writeObject (this);
    oos.flush();
    dos.finish();
    }
    [/code]

    The first method just creates a FileOutputStrea m given a file name and delegates
    the hard work to the second method. Compare this with the static load method.
    The other method wraps a DeflaterOutputS tream around the OutputStream parameter.
    This DeflaterOutputS tream is wrapped in an ObjectOutputStr eam again and the
    entire Library is written.

    Also observe that the second method doesn't close the stream for identical
    reasons as explained for the load methods (compare the logic again).

    I noticed that this article part is getting quite long again, so I see you below in
    the sequel of this article part.

    kind regards,

    Jos
Working...