Minnesota  State Archives

Preserving State Government Digital Information:
exist Pilot Project Meetings and final report

Overview

The Minnesota Historical Society (MHS) worked with developers from Syntactica on the NDIIPP pilot project. Syntactica created an integration architecture to support the business goals of this project. The first phase was to analyze business requirements and to study architectural tradeoffs. Syntactica then built proof-of-concept architecture applications that demonstrate architectural tradeoffs. One important goal was to be able to provide users with an easy to use web interface as well as with APIs (service interfaces). The diagram below outlines a draft for the project.

workflow diagram

 

For more information about the Pilot Project, please contact:

Carol Kussmann at Carol.Kussmann@mnhs.org

 

Final Reports: eXist Pilot Project August 2009-January 2010

The following documents were created to provide background information and assist with system evaluation.

January 20, 2010 All Partners Meeting

A major portion of the 2009 All Partners Meeting was used to discuss and demonstrate eXist pilot application. The All Partner Meetings webpage provides more information including presentation slides and the full meeting summary.

 

December 14, 2009 Meeting Summary

Syntactica team members and MHS team members met for to review new or modified applications including Checksums; the Uploader; Load and Stress Tester; Index Advisor; and Search. There was also discussion on the expectations for the final report and what topics will be covered in the January 2010 All Partners meeting.

 

December 7, 2009 Meeting Summary

Carol Kussmann, NDIIPP team member, was given a preview of additional applications that are being developed for the application portfolio. The Search application and a user friendly Index Advisor was covered. Details about how the applications were developed were also discussed. In addition, the first draft of the outline of the final report was reviewed.

 

November 12, 2009 Meeting Summary

NDIIPP team members were given a preview of some of the applications being developed for the application portfolio. These included the Uploader; Indexing; Index Advisor; and a discussion on the possibilities for boosting the ranking of search results. In addition, other important ideas were discussed including: ingestion, transformation, indexing, extraction, searching, scalability, and documentation of decisions.

 

October 7, 2009 Meeting Summary

To address some partners concerns, stress testing will be done on the eXist system using a virtual machine at MHS. Reliability on a system that provides access to bill and statutes must be very high at all times. To gather statistics, the system should have some performance and load testing done. Sample scripts have been created based off of the ten most common queries on bill files.

 

October 1, 2009 Meeting Summary

Syntactica attended a portion of the NDIIPP team's weekly meeting. During this time the following topics were discussed: the importance of user stories and how they can be used to show that the pilot will meet the business needs of the project; discussion about the list of current applications listed in the application portfolio; discussion of roles and role based access; discussion of authentication and checksums; and a discussion of stress testing on demo servers.

 

September 18, 2009 Meeting Summary

Syntactica presented an architectural tradeoff of four different system architectures: 1) a traditional relational database, 2) a native XML system, 3) a document search and retrieval system, and 4) a hybrid system that might use both document search and SQL for metadata.

The architecturally significant requirements previously ranked as Critical, Very High, High, and Medium by MHS staff were scored using a five point scale and then weighted based on how critical they were to the overall success of the project using 100% for Critical, 75% for Very High, 50% for High, and 25% for Medium. The total scores for each system architecture follow:

1) Relational Database: 19

2) Native XML System: 40

3) Document Management System: 24.75

4) Hybrid: 33

This was followed by a general discussion of architecture quality attributes.

 

August 26, 2009 Meeting Summary

Staff from the Minnesota Revisor's Office joined MHS and Syntactica for this meeting to discuss further system requirements. The Revisor's Office shared an outline for work to consider when developing this pilot. At least 50% of the items fall into the project's first phase. The remainder if not added into this first phase will become goals for future developments. Some of the requirements that were discussed included system documentation, acceptable data types, importing and exporting features, and search capabilities.

Syntactica created an initial list of Architecturally Significant requirements that MHS staff will review and comment on. These comments will be taken into consideration when Syntactica provides MHS with the Statement of High-Level Architectural Requirements. After being approved, this Statement of High-Level Architectural Requirements will be used by Syntactica to focus on the most important system requirements.

 

August 19, 2009 Meeting Summary

An initial meeting to discuss the overall architecture needed for the database application. Discussion of expected deliverables including a High-Level Needs Analysis; Statement of High-Level Architectural Requirements; Architectures Tradeoff Modeling Session; Proof of Architecture Application Portfolio; Application Creation and Testing; and the Architectural Evaluation Report. Many of these things need to be discussed and created before the actual development will begin. Documentation of the entire process will be very important.

 

NDIIPP Project Homepage

February 24, 2010